Analyzing Process Data from Game/Scenario-Based Tasks: An Edit Distance Approach

Autor/inn/en	Hao, Jiangang; Shu, Zhan; von Davier, Alina
Titel	Analyzing Process Data from Game/Scenario-Based Tasks: An Edit Distance Approach
Quelle	In: Journal of Educational Data Mining, 7 (2015) 1, S.33-50 (18 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2157-2100
Schlagwörter	Task Analysis; Data Analysis; Vignettes; Correlation; Scoring Rubrics; Comparative Analysis; Student Evaluation; Scores; National Competency Tests; Engineering; Literacy; Responses; Computer Games; Engineering Education; Computer Assisted Instruction; Factor Analysis; National Assessment of Educational Progress + Suchen Sie Ihr Suchwort? Aufgabenanalyse; Auswertung; Korrelation; Scoring formulas; Auswertungsbogen; Schulnote; Studentische Bewertung; Maschinenbau; Alphabetisierung; Schreib- und Lesefähigkeit; Computer game; Computerspiel; Computerspiele; Ingenieurausbildung; Computer based training; Computerunterstützter Unterricht; Faktorenanalyse
Abstract	Students' activities in game/scenario-based tasks (G/SBTs) can be characterized by a sequence of time-stamped actions of different types with different attributes. For a subset of G/SBTs in which only the order of the actions is of great interest, the process data can be well characterized as a string of characters (i.e., action string) if we encode each action name as a single character. In this article, we report our work on evaluating students' performances by comparing how far their action strings are from the action string that corresponds to the best performance, where the proximity is quantified by the edit distance between the strings. Specifically, we choose the Levenshtein distance, which is defined as the minimum number of insertions, deletions, and replacements needed to convert one character string into another. Our results show a strong correlation between the edit distances and the scores obtained from the scoring rubrics of the pump repair task from the National Assessment of Education Progress Technology and Engineering Literacy assessments, implying that the edit distance to the best performance sequence can be considered as a new feature variable that encodes information about students' proficiency, which sheds light on the value of data-driven scoring rules for test and task development and for refining the scoring rubrics. (As Provided).
Anmerkungen	International Working Group on Educational Data Mining. e-mail: jedm.editor@gmail.com; Web site: http://www.educationaldatamining.org/JEDM/index.php/JEDM/index
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01