University of Oulu

Noroozi, O., Pijeira-Díaz, H.J., Sobocinski, M. et al. Multimodal data indicators for capturing cognitive, motivational, and emotional learning processes: A systematic literature review. Educ Inf Technol 25, 5499–5547 (2020).

Multimodal data indicators for capturing cognitive, motivational, and emotional learning processes : a systematic literature review

Saved in:
Author: Noroozi, Omid1,2; Pijeira-Díaz, Héctor J.1; Sobocinski, Marta1;
Organizations: 1University of Oulu, Oulu, Finland
2Education and Learning Sciences, Wageningen University and Research, P.O. Box 8130, NL 6700 EW, Wageningen, Netherlands
3Open University of the Netherlands, 6419 AT, Heerlen, Netherlands
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 1.2 MB)
Persistent link:
Language: English
Published: Springer Nature, 2020
Publish Date: 2020-12-10


This systematic review on data modalities synthesises the research findings in terms of how to optimally use and combine such modalities when investigating cognitive, motivational, and emotional learning processes. ERIC, WoS, and ScienceDirect databases were searched with specific keywords and inclusion criteria for research on data modalities, resulting in 207 relevant publications. We provide findings in terms of target journal, country, subject, participant characteristics, educational level, foci, type of data modality, research method, type of learning, learning setting, and modalities used to study the different foci. In total, 18 data modalities were classified. For the 207 multimodal publications, 721 occurrences of modalities were observed. The most popular modality was interview followed by survey and observation. The least common modalities were heart rate variability, facial expression recognition, and screen recording. From the 207 publications, 98 focused exclusively on the cognitive aspects of learning, followed by 27 publications that only focused on motivation, while only five publications exclusively focused on emotional aspects. Only 10 publications focused on a combination of cognitive, motivational, and emotional aspects of learning. Our results plea for the increased use of objective measures, highlight the need for triangulation of objective and subjective data, and demand for more research on combining various aspects of learning. Further, rather than researching cognitive, motivational, and emotional aspects of learning separately, we encourage scholars to tap into multiple learning processes with multimodal data to derive a more comprehensive view on the phenomenon of learning.

see all

Series: Education and information technologies
ISSN: 1360-2357
ISSN-E: 1573-7608
ISSN-L: 1360-2357
Volume: 25
Pages: 5499 - 5547
DOI: 10.1007/s10639-020-10229-w
Type of Publication: A2 Review article in a scientific journal
Field of Science: 516 Educational sciences
Funding: This study was supported by the Finnish Academy grant 275440.
Academy of Finland Grant Number: 275440
Detailed Information: 275440 (Academy of Finland Funding decision)
Copyright information: © The Author(s) 2020. Open Access. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit