Facial Emotion Recognition of Deaf and Hard-of-Hearing Students for Engagement Detection Using Deep Learning

Autor/inn/en	Lasri, Imane; Riadsolh, Anouar; Elbelkacemi, Mourad
Titel	Facial Emotion Recognition of Deaf and Hard-of-Hearing Students for Engagement Detection Using Deep Learning
Quelle	In: Education and Information Technologies, 28 (2023) 4, S.4069-4092 (24 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Lasri, Imane)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1360-2357
DOI	10.1007/s10639-022-11370-4
Schlagwörter	Emotional Response; Recognition (Psychology); Nonverbal Communication; Deafness; Hearing Impairments; Students with Disabilities; Learner Engagement; Technology Uses in Education + Suchen Sie Ihr Suchwort? Emotionales Verhalten; Recognition; Wiedererkennen; Non-verbal communication; Nonverbale Kommunikation; Gehörlosigkeit; Taubstummheit; Hearing impairment; Hörbehinderung; Student; Students; Disability; Disabilities; Schüler; Schülerin; Studentin; Behinderung; Technology enhanced learning; Technology aided learning; Technologieunterstütztes Lernen
Abstract	Nowadays, facial expression recognition (FER) has drawn considerable attention from the research community in various application domains due to the recent advancement of deep learning. In the education field, facial expression recognition has the potential to evaluate students' engagement in a classroom environment, especially for deaf and hard-of-hearing students. Several works have been conducted on detecting students' engagement from facial expressions using traditional machine learning or convolutional neural network (CNN) with only a few layers. However, measuring deaf and hard-of-hearing students' engagement is yet an unexplored area for experimental research. Therefore, we propose in this study a novel approach for detecting the engagement level ('highly engaged', 'nominally engaged', and 'not engaged') from the facial emotions of deaf and hard-of-hearing students using a deep CNN (DCNN) model and transfer learning (TL) technique. A pre-trained VGG-16 model is employed and fine-tuned on the Japanese female facial expression (JAFFE) dataset and the Karolinska directed emotional faces (KDEF) dataset. Then, the performance of the proposed model is compared to seven different pre-trained DCNN models (VGG-19, Inception v3, DenseNet-121, DenseNet-169, MobileNet, ResNet-50, and Xception). On the 10-fold cross-validation case, the best-achieved test accuracies with VGG-16 are 98% and 99% on JAFFE and KDEF datasets, respectively. According to the obtained results, the proposed approach outperformed other state-of-the-art methods. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01