Using a Model of Analysts' Judgments to Augment an Item Calibration Process

Autor/inn/en	Hauser, Carl; Thum, Yeow Meng; He, Wei; Ma, Lingling
Titel	Using a Model of Analysts' Judgments to Augment an Item Calibration Process
Quelle	In: Educational and Psychological Measurement, 75 (2015) 5, S.826-849 (24 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0013-1644
DOI	10.1177/0013164414559775
Schlagwörter	Test Items; Item Response Theory; Research Methodology; Decision Making; Models; Statistics; Graphs; Prediction; Classification; Accuracy; Evidence; Regression (Statistics); Psychometrics; Computer Assisted Testing; Mathematics Achievement; Reading Achievement; Science Achievement; Language Usage; Elementary Secondary Education; Measurement Techniques; Probability; Error Patterns + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Item-Response-Theorie; Research method; Forschungsmethode; Decision-making; Entscheidungsfindung; Analogiemodell; Statistik; Grafische Darstellung; Vorhersage; Classification system; Klassifikation; Klassifikationssystem; Evidenz; Regression; Regressionsanalyse; Psychometry; Psychometrie; Mathmatics sikills; Mathmatics achievement; Mathematical ability; Mathematische Kompetenz; Leseleistung; Sprachgebrauch; Messtechnik; Wahrscheinlichkeitsrechnung; Wahrscheinlichkeitstheorie; Fehlertyp
Abstract	When conducting item reviews, analysts evaluate an array of statistical and graphical information to assess the fit of a field test (FT) item to an item response theory model. The process can be tedious, particularly when the number of human reviews (HR) to be completed is large. Furthermore, such a process leads to decisions that are susceptible to human errors. A key finding from behavioral decision-making research has shown that a parametric model of human decision making often outperforms the decision maker himself. We exploit this finding by seeking a model to mimic how analysts integrate FT item level statistics and graphical performance plots to predict the analyst's assignment of the item's status. The procedure suggests a set of rules that achieves a desired level of classification accuracy, separating situations in which the evidence supports firm decisions from those situations that would likely benefit from HRs. Implementation of the decision rules accounts for an estimated 65% reduction in calibrations requiring HRs. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01