When a Machine Detects Student Reasoning: A Review of Machine Learning-Based Formative Assessment of Mechanistic Reasoning

Autor/inn/en	Martin, Paul P.; Graulich, Nicole
Titel	When a Machine Detects Student Reasoning: A Review of Machine Learning-Based Formative Assessment of Mechanistic Reasoning
Quelle	In: Chemistry Education Research and Practice, 24 (2023) 2, S.407-427 (21 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Martin, Paul P.) ORCID (Graulich, Nicole)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
Schlagwörter	Artificial Intelligence; Critical Thinking; Logical Thinking; Student Evaluation; Formative Evaluation; Chemistry; Educational Research; Science Process Skills; Scoring Rubrics; Design; Algorithms + Suchen Sie Ihr Suchwort? Künstliche Intelligenz; Kritisches Denken; Schulnote; Studentische Bewertung; Chemie; Bildungsforschung; Pädagogische Forschung; Scoring formulas; Auswertungsbogen; Algorithm; Algorithmus
Abstract	In chemistry, reasoning about the underlying mechanisms of observed phenomena lies at the core of scientific practices. The process of uncovering, analyzing, and interpreting mechanisms for explanations and predictions requires a specific kind of reasoning: mechanistic reasoning. Several frameworks have already been developed that capture the aspects of mechanistic reasoning to support its formative assessment. However, evaluating mechanistic reasoning in students' open responses is a time- and resource-intense, complex, and challenging task when performed by hand. Emerging technologies like machine learning (ML) can automate and advance the formative assessment of mechanistic reasoning. Due to its usefulness, ML has already been applied to assess mechanistic reasoning in several research projects. This review focuses on 20 studies dealing with ML in chemistry education research capturing mechanistic reasoning. We developed a six-category framework based on the evidence-centered design (ECD) approach to evaluate these studies in terms of "pedagogical purpose," "rubric design," "construct assessment," "validation approaches," "prompt structure," and "sample heterogeneity." Contemporary effective practices of ML-based formative assessment of mechanistic reasoning in chemistry education are emphasized to guide future projects by these practices and to overcome challenges. Ultimately, we conclude that ML has advanced replicating, automating, and scaling human scoring, while it has not yet transformed the quality of evidence drawn from formative assessments. (As Provided).
Anmerkungen	Royal Society of Chemistry. Thomas Graham House, Science Park, Milton Road, Cambridge, CB4 0WF, UK. Tel: +44-1223 420066; Fax: +44-1223 423623; e-mail: cerp@rsc.org; Web site: http://www.rsc.org/cerp
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01