A Comparison of Score Aggregation Methods for Unidimensional Tests on Different Dimensions. Research Report. ETS RR-18-01

Autor/inn/en	Fu, Jianbin; Feng, Yuling
Titel	A Comparison of Score Aggregation Methods for Unidimensional Tests on Different Dimensions. Research Report. ETS RR-18-01
Quelle	In: ETS Research Report Series, (2018), (18 Seiten)Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2330-8516
Schlagwörter	Comparative Analysis; Scores; Correlation; Standardized Tests; Test Length; Simulation; Item Response Theory; Raw Scores; Factor Analysis; Models; Test Items + Suchen Sie Ihr Suchwort? Korrelation; Standadised tests; Standardisierter Test; Simulation program; Simulationsprogramm; Item-Response-Theorie; Faktorenanalyse; Analogiemodell; Test content; Testaufgabe
Abstract	In this study, we propose aggregating test scores with unidimensional within-test structure and multidimensional across-test structure based on a 2-level, 1-factor model. In particular, we compare 6 score aggregation methods: average of standardized test raw scores (M1), regression factor score estimate of the 1-factor model based on the correlation matrix of test raw scores (M2), overall ability from a unidimensional generalized partial credit model (GPCM) based on the items from all tests (M3), average of ability estimates from individual tests based on GPCM (M4), regression factor score of the 1-factor model based on the correlation matrix of ability estimates from individual tests based on GPCM (M5), and general ability from the testlet model (M6). The 4 design factors considered in the simulation study are ability correlation between tests (0.3, 0.5, 0.7, 0.8, and 0.9), test length (10, 20, 30, and 60 items), number of tests (2 and 4), and factor loading distribution (equal and unequal). The comparisons are also conducted on a real test data set with 2 tests. On the basis of the results, M1 and M4 are recommended for 2 tests, and M2, M5, and M6 are recommended for 3 or more tests. Several issues regarding attaining aggregate score reliability for intended uses and score aggregation types distinguished by test dimensionality are discussed, and practical suggestions for score aggregation are provided. (As Provided).
Anmerkungen	Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01