Dissecting Knowledge, Guessing, and Blunder in Multiple Choice Assessments

Autor/inn/en	Abu-Ghazalah, Rashid M.; Dubins, David N.; Poon, Gregory M. K.
Titel	Dissecting Knowledge, Guessing, and Blunder in Multiple Choice Assessments
Quelle	In: Applied Measurement in Education, 36 (2023) 1, S.80-98 (19 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Poon, Gregory M. K.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0895-7347
DOI	10.1080/08957347.2023.2172017
Schlagwörter	Guessing (Tests); Multiple Choice Tests; Probability; Models; Error Patterns; Responses; Item Analysis; Test Items; Undergraduate Students; Biotechnology; Bayesian Statistics; Scores; Self Esteem; Knowledge Level; Test Construction; Foreign Countries; Canada + Suchen Sie Ihr Suchwort? Erraten; Multiple choice examinations; Multiple-choice tests, Multiple-choice examinations; Multiple-Choice-Verfahren; Wahrscheinlichkeitsrechnung; Wahrscheinlichkeitstheorie; Analogiemodell; Fehlertyp; Itemanalyse; Test content; Testaufgabe; Biotechnologie; Self-esteem; Selbstaufmerksamkeit; Wissensbasis; Testaufbau; Ausland; Kanada
Abstract	Multiple choice results are inherently probabilistic outcomes, as correct responses reflect a combination of knowledge and guessing, while incorrect responses additionally reflect blunder, a confidently committed mistake. To objectively resolve knowledge from responses in an MC test structure, we evaluated probabilistic models that explicitly account for guessing, knowledge, and blunder using eight assessments (>9,000 responses) from an undergraduate biotechnology curriculum. A Bayesian implementation of the models, aimed at assessing their robustness to prior beliefs in examinee knowledge, showed that explicit estimators of knowledge are markedly sensitive to prior beliefs with scores as sole input. To overcome this limitation, we examined self-ranked confidence as a proxy knowledge indicator. For our test set, three levels of confidence resolved test performance. Responses rated as least confident were correct more frequently than expected from random selection, reflecting partial knowledge, but were balanced by blunder among the most confident responses. By translating evidence-based guessing and blunder rates to pass marks that statistically qualify a desired level of examinee knowledge, our approach finds practical utility in test analysis and design. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01