University of Oulu

Ferdinando, H. and Alasaarela, E. (2018). Enhancement of Emotion Recogniton using Feature Fusion and the Neighborhood Components Analysis. In Proceedings of the 7th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-276-9, pages 463-469. DOI: 10.5220/0006642904630469

Enhancement of emotion recogniton using feature fusion and the neighborhood components analysis

Saved in:
Author: Ferdinando, Hany1,2; Alasaarela, Esko1
Organizations: 1Health and Wellness Measurement, OPEM unit, University of Oulu, Oulu, Finland
2Department of Electrical Engineering, Petra Christian University, Surabaya, Indonesia
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 2.4 MB)
Persistent link:
Language: English
Published: SCITEPRESS Science And Technology Publications, 2018
Publish Date: 2020-02-26


Feature fusion is a common approach to improve the accuracy of the system. Several attemps have been made using this approach on the Mahnob-HCI database for affective recognition, achieving 76% and 68% for valence and arousal respectively as the highest achievements. This study aimed to improve the baselines for both valence and arousal using feature fusion of HRV-based, which used the standard Heart Rate Variability analysis, standardized to mean/standard deviation and normalized to [-1,1], and cvxEDA-based feature, calculated based on a convex optimization approach, to get the new baselines for this database. The selected features, after applying the sequential forward floating search (SFFS), were enhanced by the Neighborhood Component Analysis and fed to kNN classifier to solve 3-class classification problem, validated using leave-one-out (LOO), leave-one-subject-out (LOSO), and 10-fold cross validation methods. The standardized HRV-based features were not selected during the SFFS method, leaving feature fusion from normalized HRV-based and cvxEDA-based features only. The results were compared to previous studies using both single- and multi-modality. Applying the NCA enhanced the features such that the performances in valence set new baselines: 82.4% (LOO validation), 79.6% (10-fold cross validation), and 81.9% (LOSO validation), enhanced the best achievement from both single- and multi-modality. For arousal, the performances were 78.3%, 78.7%, and 77.7% for LOO, LOSO, and 10-fold cross validations respectively. They outperformed the best achievement using feature fusion but could not enhance the performance in single-modality study using cvxEDA-based feature. Some future works include utilizing other feature extraction methods and using more sophisticated classifier other than the simple kNN.

see all

ISBN Print: 978-989-758-276-9
Pages: 463 - 469
DOI: 10.5220/0006642904630469
Host publication: Proceedings of the 7th International Conference on Pattern Recognition Applications and Methods (ICPRAM 2018)
Host publication editor: Fred, A.
di Baja, G.S.
De Marsico, M.
Conference: International Conference on Pattern Recognition Applications and Methods
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
217 Medical engineering
Funding: This research was supported by the Finnish Cultural Foundation, Northern Ostrobothnia Regional Fund 2017.
Copyright information: © 2018 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved. CC-BY-NC-ND 4.0 license