Multidimensional CAT Item Selection Methods for Domain Scores and Composite Scores: Theory and Applications

Autor/in	Yao, Lihua
Titel	Multidimensional CAT Item Selection Methods for Domain Scores and Composite Scores: Theory and Applications
Quelle	In: Psychometrika, 77 (2012) 3, S.495-523 (29 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0033-3123
DOI	10.1007/s11336-012-9265-5
Schlagwörter	Item Banks; Test Length; Simulation; Adaptive Testing; Population Distribution; Test Reliability; Item Analysis; Error of Measurement; Item Response Theory; Comparative Analysis; Theories; Mathematics Education; Computer Assisted Testing + Suchen Sie Ihr Suchwort? Simulation program; Simulationsprogramm; Demographical distribution; Bevölkerungsverteilung; Testreliabilität; Itemanalyse; Messfehler; Item-Response-Theorie; Theory; Theorie; Mathematische Bildung
Abstract	Multidimensional computer adaptive testing (MCAT) can provide higher precision and reliability or reduce test length when compared with unidimensional CAT or with the paper-and-pencil test. This study compared five item selection procedures in the MCAT framework for both domain scores and overall scores through simulation by varying the structure of item pools, the population distribution of the simulees, the number of items selected, and the content area. The existing procedures such as Volume (Segall in "Psychometrika" 61:331-354, 1996), Kullback-Leibler information (Veldkamp & van der Linden in "Psychometrika" 67:575-588, 2002), Minimize the error variance of the linear combination (van der Linden in "J. Educ. Behav. Stat." 24:398-412, 1999), and Minimum Angle (Reckase in "Multidimensional item response theory," Springer, New York, 2009) are compared to a new procedure, Minimize the error variance of the composite score with the optimized weight, proposed for the first time in this study. The intent is to find an item selection procedure that yields higher precisions for both the domain and composite abilities and a higher percentage of selected items from the item pool. The comparison is performed by examining the absolute bias, "correlation," "test reliability," "time used," and "item usage." Three sets of item pools are used with the item parameters estimated from real live CAT data. Results show that Volume and Minimum Angle performed similarly, balancing information for all content areas, while the other three procedures performed similarly, with a high precision for both domain and overall scores when selecting items with the required number of items for each domain. The new item selection procedure has the highest percentage of "item usage." Moreover, for the overall score, it produces similar or even better results compared to those from the method that selects items favoring the general dimension using the general model (Segall in "Psychometrika: 66:79-, 2001); the general dimension method has low precision for the domain scores. In addition to the simulation study, the mathematical theories for certain procedures are derived. The theories are confirmed by the simulation applications. (As Provided).
Anmerkungen	Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: service-ny@springer.com; Web site: http://www.springerlink.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10