Assessing Argumentation Using Machine Learning and Cognitive Diagnostic Modeling

Autor/inn/en	Zhai, Xiaoming; Haudek, Kevin C.; Ma, Wenchao
Titel	Assessing Argumentation Using Machine Learning and Cognitive Diagnostic Modeling
Quelle	In: Research in Science Education, 53 (2023) 2, S.405-424 (20 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Zhai, Xiaoming) ORCID (Haudek, Kevin C.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0157-244X
DOI	10.1007/s11165-022-10062-w
Schlagwörter	Persuasive Discourse; Artificial Intelligence; Cognitive Measurement; Diagnostic Tests; Models; Algorithms; Writing (Composition); Science Process Skills; Thinking Skills; Elementary School Students; Middle School Students; Scores + Suchen Sie Ihr Suchwort? Persuasion; Persuasive Kommunikation; Künstliche Intelligenz; Diagnostic test; Diagnostischer Test; Analogiemodell; Algorithm; Algorithmus; Schreibübung; Denkfähigkeit; Middle school; Middle schools; Student; Students; Mittelschule; Mittelstufenschule; Schüler; Schülerin
Abstract	In this study, we developed machine learning algorithms to automatically score students' written arguments and then applied the cognitive diagnostic modeling (CDM) approach to examine students' cognitive patterns of scientific argumentation. We abstracted three types of skills (i.e., attributes) critical for successful argumentation practice: making claims, using evidence, and providing warrants. We developed 19 constructed response items, with each item requiring multiple cognitive skills. We collected responses from 932 students in Grades 5 to 8 and developed machine learning algorithmic models to automatically score their responses. We then applied CDM to analyze their cognitive patterns. Results indicate that machine scoring achieved the average machine-human agreements of Cohen's [kappa] = 0.73, SD = 0.09. We found that students were clustered in 21 groups based on their argumentation performance, each revealing a different cognitive pattern. Within each group, students showed different abilities regarding making claims, using evidence, and providing warrants to justify how the evidence supports a claim. The 9 most frequent groups accounted for more than 70% of the students in the study. Our in-depth analysis of individual students suggests that students with the same total ability score might vary in the specific cognitive skills required to accomplish argumentation. This result illustrates the advantage of CDM in assessing the fine-grained cognition of students during argumentation practices in science and other scientific practices. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01