Multimodal Data Fusion to Track Students' Distress during Educational Gameplay

Autor/inn/en	Moon, Jewoong; Ke, Fengfeng; Sokolikj, Zlatko; Dahlstrom-Hakki, Ibrahim
Titel	Multimodal Data Fusion to Track Students' Distress during Educational Gameplay
Quelle	In: Journal of Learning Analytics, 9 (2022) 3, S.75-87 (13 Seiten)Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei Verfügbarkeit
Zusatzinformation	ORCID (Moon, Jewoong) ORCID (Ke, Fengfeng) ORCID (Sokolikj, Zlatko) ORCID (Dahlstrom-Hakki, Ibrahim)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
Schlagwörter	Learning Analytics; Stress Variables; Educational Games; Middle School Students; Data Collection; Prediction; Identification; Cognitive Processes; Affective Behavior; Video Technology; Videoconferencing; Nonverbal Communication; Problem Solving + Suchen Sie Ihr Suchwort? Educational game; Lernspiel; Middle school; Middle schools; Student; Students; Mittelschule; Mittelstufenschule; Schüler; Schülerin; Data capture; Datensammlung; Vorhersage; Identifikation; Identifizierung; Cognitive process; Kognitiver Prozess; Affective disturbance; Active behaviour; Affektive Störung; Non-verbal communication; Nonverbale Kommunikation; Problemlösen
Abstract	Using multimodal data fusion techniques, we built and tested prediction models to track middle-school student distress states during educational gameplay. We collected and analyzed 1,145 data instances, sampled from a total of 31 middle-school students' audio- and video-recorded gameplay sessions. We conducted data wrangling with student gameplay data from multiple data sources, such as individual facial expression recordings and gameplay logs. Using supervised machine learning, we built and tested candidate classifiers that yielded an estimated probability of distress states. We then conducted confidence-based data fusion that averaged the estimated probability scores from the unimodal classifiers with a single data source. The results of this study suggest that the classifier with multimodal data fusion improves the performance of tracking distress states during educational gameplay, compared to the performance of unimodal classifiers. The study finding suggests the feasibility of multimodal data fusion in developing game-based learning analytics. Also, this study proposes the benefits of optimizing several methodological means for multimodal data fusion in educational game research. (As Provided).
Anmerkungen	Society for Learning Analytics Research. 121 Pointe Marsan, Beaumont, AB T4X 0A2, Canada. Tel: +61-429-920-838; e-mail: info@solaresearch.org; Web site: https://learning-analytics.info/index.php/JLA/index
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01