University of Oulu

Z. Xia, W. Peng, H. -Q. Khor, X. Feng and G. Zhao, "Revealing the Invisible With Model and Data Shrinking for Composite-Database Micro-Expression Recognition," in IEEE Transactions on Image Processing, vol. 29, pp. 8590-8605, 2020, doi: 10.1109/TIP.2020.3018222

Revealing the invisible with model and data shrinking for composite-database micro-expression recognition

Saved in:
Author: Xia, Zhaoqiang1,2; Peng, Wei3; Khor, Huai-Qian3;
Organizations: 1School of Electronics and Information, Northwestern Polytechnical University
2Center for Machine Vision and Signal Analysis, University of Oulu
3Center for Machine Vision and Signal Analysis, University of Oulu, 90014 Oulu, Finland
4School of Electronics and Information, Northwestern Polytechnical University, 710129 Shaanxi
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.9 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2020
Publish Date: 2020-11-10


Composite-database micro-expression recognition is attracting increasing attention as it is more practical for real-world applications. Though the composite database provides more sample diversity for learning good representation models, the important subtle dynamics are prone to disappearing in the domain shift such that the models greatly degrade their performance, especially for deep models. In this article, we analyze the influence of learning complexity, including input complexity and model complexity, and discover that the lower-resolution input data and shallower-architecture model are helpful to ease the degradation of deep models in composite-database task. Based on this, we propose a recurrent convolutional network (RCN) to explore the shallower-architecture and lower-resolution input data, shrinking model and input complexities simultaneously. Furthermore, we develop three parameter-free modules (i.e., wide expansion, shortcut connection and attention unit) to integrate with RCN without increasing any learnable parameters. These three modules can enhance the representation ability in various perspectives while preserving not-very-deep architecture for lower-resolution data. Besides, three modules can further be combined by an automatic strategy (a neural architecture search strategy) and the searched architecture becomes more robust. Extensive experiments on the MEGC2019 dataset (composited of existing SMIC, CASME II and SAMM datasets) have verified the influence of learning complexity and shown that RCNs with three modules and the searched combination outperform the state-of-the-art approaches.

see all

Series: IEEE transactions on image processing
ISSN: 1057-7149
ISSN-E: 1941-0042
ISSN-L: 1057-7149
Volume: 29
Pages: 8590 - 8605
DOI: 10.1109/TIP.2020.3018222
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Funding: This work is partly supported by the National Natural Science Foundation of China (Nos. 61702419, 61772419),by the Natural Science Basic Research Plan in Shaanxi Province of China (No. 2018JQ6090), Tekes Fidipro program (No. 1849/31/2015) and Business Finland project (No. 3116/31/2017), Infotech Oulu, Academy of Finland ICT 2023 project (313600). As well, the authors wish to acknowledge the CSCIT Center for Science, Finland, for computational resources.
Academy of Finland Grant Number: 313600
Detailed Information: 313600 (Academy of Finland Funding decision)
Copyright information: © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.