You Are What You Click: Using Machine Learning to Model Trace Data for Psychometric Measurement

Autor/inn/en	Landers, Richard N.; Auer, Elena M.; Mersy, Gabriel; Marin, Sebastian; Blaik, Jason
Titel	You Are What You Click: Using Machine Learning to Model Trace Data for Psychometric Measurement
Quelle	In: International Journal of Testing, 22 (2022) 3-4, S.243-263 (21 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Landers, Richard N.) ORCID (Auer, Elena M.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1530-5058
DOI	10.1080/15305058.2022.2134394
Schlagwörter	Artificial Intelligence; Data Collection; Psychometrics; Data Science; Computer Use; Computer Peripherals; Algorithms; Engineering; Measurement + Suchen Sie Ihr Suchwort? Künstliche Intelligenz; Data capture; Datensammlung; Psychometry; Psychometrie; Algorithm; Algorithmus; Maschinenbau; Messverfahren
Abstract	Assessment trace data, such as mouse positions and their timing, offer interesting and provocative reflections of individual differences yet are currently underutilized by testing professionals. In this article, we present a 10-step procedure to maximize the probability that a trace data modeling project will be successful: (1) grounding the project in psychometric theory, (2) building technical infrastructure to collect trace data, (3) designing a useful developmental validation study, (4) using a holdout validation approach with collected data, (5) using exploratory analysis to conduct meaningful feature engineering, (6) identifying useful machine learning algorithms to predict a thoughtfully chosen criterion, (7) engineering a machine learning model with meaningful internal cross-validation and hyperparameter selection, (8) conducting model diagnostics to assess if the resulting model is overfitted, underfitted, or within acceptable tolerance, and (9) testing the success of the final model in meeting conceptual, technical, and psychometric goals. If deemed successful, trace data model predictions could then be engineered into decision-making systems. We present this framework within the broader view of psychometrics, exploring the challenges of developing psychometrically valid models using such complex data with much weaker trait signals than assessment developers have typically attempted to model. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01