Lukas Stappen, Alice Baird, Lukas Christ, Lea Schumann, Benjamin Sertolli, Eva-Maria Meßner, Erik Cambria, Guoying Zhao, and Björn W. Schuller. 2021. The MuSe 2021 Multimodal Sentiment Analysis Challenge: Sentiment, Emotion, Physiological-Emotion, and Stress. Proceedings of the 2nd on Multimodal Sentiment Analysis Challenge. Association for Computing Machinery, New York, NY, USA, 5–14. DOI:https://doi.org/10.1145/3475957.3484450
The MuSe 2021 multimodal sentiment analysis challenge : sentiment, emotion, physiological-emotion, and stress
|Author:||Stappen, Lukas1; Baird, Alice1; Christ, Lukas1;|
1University of Augsburg, Augsburg, Germany
2University of Ulm, Ulm, Germany
3Nanyang Technological University, Singapore
4University of Oulu, Oulu, Finland
5Imperial College London, London, United Kingdom
|Online Access:||PDF Full Text (PDF, 1.5 MB)|
|Persistent link:|| http://urn.fi/urn:nbn:fi-fe2022022420753
Association for Computing Machinery,
|Publish Date:|| 2022-02-24
Multimodal Sentiment Analysis (MuSe) 2021 is a challenge focusing on the tasks of sentiment and emotion, as well as physiological-emotion and emotion-based stress recognition through more comprehensively integrating the audio-visual, language, and biological signal modalities. The purpose of MuSe 2021 is to bring together communities from different disciplines; mainly, the audio-visual emotion recognition community (signal-based), the sentiment analysis community (symbol-based), and the health informatics community. We present four distinct sub-challenges: MuSe-Wilder and MuSe-Stress which focus on continuous emotion (valence and arousal) prediction; MuSe-Sent, in which participants recognise five classes each for valence and arousal; and MuSe-Physio, in which the novel aspect of ‘physiological-emotion’ is to be predicted. For this year’s challenge, we utilise the MuSe-CaR dataset focusing on user-generated reviews and introduce the Ulm-TSST dataset, which displays people in stressful depositions. This paper also provides detail on the state-of-the-art feature sets extracted from these datasets for utilisation by our baseline model, a Long Short-Term Memory-Recurrent Neural Network. For each sub-challenge, a competitive baseline for participants is set; namely, on test, we report a Concordance Correlation Coefficient (CCC) of .4616 CCC for MuSe-Wilder; .5088 CCC for MuSe-Stress, and .4908 CCC for MuSe-Physio. For MuSe-Sent an F1 score of 32.82% is obtained.
|Pages:||5 - 14|
2nd Multimodal Sentiment Analysis Challenge and Workshop, MuSe 2021, held in conjunction with the ACM Multimedia 2021
Multimodal Sentiment Analysis Challenge and Workshop
|Type of Publication:||
A4 Article in conference proceedings
|Field of Science:||
113 Computer and information sciences
This project has received funding from the European Union’s Horizon 2020 research and the DFG’s Reinhart Koselleck project No. 442218748 (AUDI0NOMOUS). We thank the sponsors of the Challenge, the BMW Group, and audEERING.
© 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM.