Hierarchical IRT Examination of Isomorphic Equivalence of Complex Constructed Response Tasks.

Autor/inn/en	Williamson, David M.; Johnson, Matthew S.; Sinharay, Sandip; Bejar, Isaac I.
Institution	Educational Testing Service, Princeton, NJ.
Titel	Hierarchical IRT Examination of Isomorphic Equivalence of Complex Constructed Response Tasks.
Quelle	(2002), (18 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Bayesian Statistics; Constructed Response; Estimation (Mathematics); Evaluation Methods; High Stakes Tests; Item Response Theory; Markov Processes; Monte Carlo Methods; Test Construction + Suchen Sie Ihr Suchwort? Estimation; Mathematics; Schätzung; Item-Response-Theorie; Markowscher Prozess; Monte-Carlo-Methode; Testaufbau
Abstract	This paper explores the application of a technique for hierarchical item response theory (IRT) calibration of complex constructed response tasks that has promise both as a calibration tool and as a means of evaluating the isomorphic equivalence of complex constructed response tasks. Isomorphic tasks are explicitly and rigorously designed to be highly similar in domain-relevant characteristics and evaluation standards. A related task model was used in which each item was modeled with a separate item response function, but the isomorphic tasks were related through a hierarchical model. The model was implemented in software that conducted Bayesian Markov Chain Monte Carlo (MCMC) estimation to estimate the joint posterior of all model parameters by integrating over the posterior distribution of model parameters given the data. The study analyzed operational data from a high-stakes assessment consisting of a number of complex constructed response tasks. The MCMC estimation procedure was conducted through 100,000 iterations. The item characteristic curves (ICCs) for the six isomorphic families were determined. In general, the families of isomorphic tasks showed considerable similarity in the item response functions for their respective members as well as for the family response function for the isomorphic set. Results suggest that efforts to construct complex constructed response tasks that are isomorphic equivalent tasks can range somewhat in their degree of success, with some being consistently equivalent, some being more variable, and others being largely consistent but with notable deviations. (SLD)
Erfasst von	ERIC (Education Resources Information Center), Washington, DC