University of Oulu

Nguyen, A., Järvelä, S., Wang, Y., & Róse, C. (2022). Exploring socially shared regulation with an AI deep learning approach using multimodal data. In C. Chinn, E. Tan, C. Chan & Y. Kali (Eds.), Proceedings of International Conferences of Learning Sciences (ICLS) (pp. 527-534). Hiroshima, Japan. Retrieved from

Exploring socially shared regulation with an AI deep learning approach using multimodal data

Saved in:
Author: Nguyen, Andy1; Järvelä, Sanna1; Wang, Yansen2;
Organizations: 1University of Oulu
2Microsoft Research Lab (MSRA) Shanghai
3Carnegie Mellon University
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 0.4 MB)
Persistent link:
Language: English
Published: International Society of the Learning Sciences, 2022
Publish Date: 2022-11-22


Socially shared regulation of learning (SSRL) is essential for the success of collaborative learning, yet learners often neglect needed regulation while facing challenges. In order to provide targeted support when needed, it is critical to identify the precise events that trigger regulation. Multimodal collaborative learning data may offer opportunities for this. This study aims to lay such a foundation by exploring the potential for using machine-learned models trained on multimodal data, including electrodermal activities (EDA), speech, and video, to detect the presence of SSRL-relevant process-level indicators in successful and less successful groups. The study involves thirty groups of secondary students (N=94) working collaboratively in five physics lessons. Considering the demonstrated positive results of machine-learned models, the advantages and limitations of the technical approach are discussed, and further development directions are suggested.

see all

Series: International Conference of the Learning Sciences
ISSN: 1814-9316
ISSN-E: 1819-0138
ISSN-L: 1814-9316
ISBN: 978-1-7373306-5-3
Pages: 527 - 534
Host publication: Proceedings of the 16th international conference of the learning sciences - ICLS 2022
Host publication editor: Chinn, Clark
Tan, Edna
Chan, Carol
Kali, Yael
Conference: International conference of the learning sciences
Type of Publication: A4 Article in conference proceedings
Field of Science: 516 Educational sciences
113 Computer and information sciences
Funding: This work was funded in part by NSF grants IIS 1822831 and 1917955 and funding from Microsoft, Finnish Academy project no. 32438 and Oulu University Eudaimonia TRIGGER project funding.
Copyright information: © 2022 International Society of the Learning Sciences, Inc. [ISLS].