Adaptive modality distillation for separable multimodal sentiment analysis
|Author:||Peng, Wei1; Hong, Xiaopeng2; Zhao, Guoying1|
1University of Oulu
2Xian Jiaotong University
|Online Access:||PDF Full Text (PDF, 0.8 MB)|
|Persistent link:|| http://urn.fi/urn:nbn:fi-fe202104099805
Institute of Electrical and Electronics Engineers,
|Publish Date:|| 2021-04-09
Multimodal sentiment analysis has increasingly attracted attention since with the arrival of complementary data streams, it has great potential to improve and go beyond unimodal sentiment analysis. In this paper, we present an efficient separable multimodal learning method to deal with the tasks with modality missing issue. In this method, the multimodal tensor is utilized to guide the evolution of each separated modality representation. To save the computational expense, Tucker decomposition is introduced, which leads to a general extension of the low-rank tensor fusion method with more modality interactions. The method, in turn, enhances our modality distillation processing. Comprehensive experiments on three popular multimodal sentiment analysis datasets, CMU-MOSI, POM, and IEMOCAP, show a superior performance especially when only partial modalities are available.
IEEE intelligent systems
|Pages:||1 - 9|
|Type of Publication:||
A1 Journal article – refereed
|Field of Science:||
113 Computer and information sciences
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists,or reuse of any copyrighted component of this work in other works.