Covariates of the Rating Process in Hierarchical Models for Multiple Ratings of Test Items

Autor/inn/en	Mariano, Louis T.; Junker, Brian W.
Titel	Covariates of the Rating Process in Hierarchical Models for Multiple Ratings of Test Items
Quelle	In: Journal of Educational and Behavioral Statistics, 32 (2007) 3, S.287-314 (28 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1076-9986
DOI	10.3102/1076998606298033
Schlagwörter	Test Items; Item Response Theory; Rating Scales; Scoring; Interrater Reliability; Matrices; Student Evaluation; Bayesian Statistics; Models; Measurement Techniques; Monte Carlo Methods + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Item-Response-Theorie; Rating-Skala; Bewertung; Interrater-Reliabilität; Matrizenrechnung; Schulnote; Studentische Bewertung; Analogiemodell; Messtechnik; Monte-Carlo-Methode
Abstract	When constructed response test items are scored by more than one rater, the repeated ratings allow for the consideration of individual rater bias and variability in estimating student proficiency. Several hierarchical models based on item response theory have been introduced to model such effects. In this article, the authors demonstrate how these models may be extended to include covariates of the rating process. For example, how do features of an essay grader's training affect his or her performance? The authors show how to include covariates by embedding a linear model at appropriate levels of the model hierarchy. Depending on the level, such covariates may be thought of as determining fixed effects or random effects on the rating process. The authors also discuss the appropriate design matrix for such covariates, discuss how to incorporate needed identifiability constraints, and illustrate the methods using data from a rating study of a student assessment. (Contains 9 tables.) (Author).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10