Xiaopeng Hong, Wei Peng, Mehrtash Harandi, Ziheng Zhou, Matti Pietikäinen, and Guoying Zhao. 2019. Characterizing Subtle Facial Movements via Riemannian Manifold. ACM Trans. Multimedia Comput. Commun. Appl. 15, 3s, Article 94 (December 2019), 24 pages. DOI:https://doi.org/10.1145/3342227
Characterizing subtle facial movements via Riemannian manifold
|Author:||Hong, Xiaopeng1,2; Peng, Wei2; Harandi, Mehrtash3,4;|
1Xi’an Jiaotong University
2University of Oulu
|Online Access:||PDF Full Text (PDF, 3 MB)|
|Persistent link:|| http://urn.fi/urn:nbn:fi-fe2020042322159
Association for Computing Machinery,
|Publish Date:|| 2020-04-23
Characterizing subtle facial movements from videos is one of the most intensive topics in computer vision research. It is, however, challenging, since (1) the intensity of subtle facial muscle movement is usually low, (2) the duration may be transient, and (3) datasets containing spontaneous subtle movements with reliable annotations are painful to obtain and often of small sizes.
This article is targeted at addressing these problems for characterizing subtle facial movements from both the aspects of motion elucidation and description. First, we propose an efficient method for elucidating hidden and repressed movements to make them easier to get noticed. We explore the feasibility of linearizing motion magnification and temporal interpolation, which is obscured by the architecture of existing methods. On this basis, we propose a consolidated framework, termed MOTEL, to expand temporal duration and amplify subtle facial movements simultaneously. Second, we make our contribution to dynamic description. One major challenge is to capture the intrinsic temporal variations caused by movements and omit extrinsic ones caused by different individuals and various environments. To diminish the influences of such extrinsic diversity, we propose the tangent delta descriptor to characterize the dynamics of short-term movements using the differences between points on the tangent spaces to the manifolds, rather than the points themselves. We then relax the trajectory-smooth assumption of the conventional manifold-based trajectory modeling methods and incorporate the tangent delta descriptor with the sequential inference approaches to cover the period of facial movements. The proposed motion modeling approach is validated by a series of experiments on publicly available datasets in the tasks of micro-expression recognition and visual speech recognition.
ACM transactions on multimedia computing, communications and applications
|Type of Publication:||
A1 Journal article – refereed
|Field of Science:||
113 Computer and information sciences
We express deep gratitude to National Basic Research Program of China (Grant No. 2015CB351705), National Major Project (Grant No. 2017YFC0803905), and Natural Science Foundation of China under Contracts No. 61772419, No. 61572205, and No. 61601362, the Academy of Finland ICT 2023 (Project No. 313600), Infotech, Tekes Fidipro Program (Grant No. 1849/31/2015), Tekes project (Grant No. 3116/31/2017).
|Academy of Finland Grant Number:||
313600 (Academy of Finland Funding decision)
© 2019 Association for Computing Machinery. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM Transactions on Multimedia Computing, Communications, and Applications, https://doi.org/10.1145/3342227.