Matching the Judgmental Task with Standard Setting Panelist Expertise: The Item-Descriptor (ID) Matching Method

Autor/inn/en	Ferrara, Steve; Perie, Marianne; Johnson, Eugene
Titel	Matching the Judgmental Task with Standard Setting Panelist Expertise: The Item-Descriptor (ID) Matching Method
Quelle	In: Journal of Applied Testing Technology, 9 (2008) 1, S.1-22 (22 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
Schlagwörter	Stellungnahme; Test Results; Test Content; Testing Programs; Educational Testing; Standard Setting (Scoring); Program Effectiveness; Cutting Scores; Psychometrics; Educational Assessment; Evaluation Methods; Item Response Theory; Computer Assisted Testing + Suchen Sie Ihr Suchwort? Testaufgabe; Bewertungsskala; Psychometry; Psychometrie; Education; assessment; Bewertungssystem; Item-Response-Theorie
Abstract	Psychometricians continue to introduce new approaches to setting cut scores for educational assessments in an attempt to improve on current methods. In this paper we describe the Item-Descriptor (ID) Matching method, a method based on IRT item mapping. In ID Matching, test content area experts match items (i.e., their judgments about the knowledge and skills required to respond to an item) to the knowledge and skills described in performance level descriptors that are used for reporting test results. We argue that the cognitive-judgmental task of matching item response requirements to performance level descriptors is aligned closely with the experience and expertise of standard setting panelists, who are typically classroom teachers and other content area experts. Unlike other popular standard setting methods, ID Matching does not require panelists to make error-prone probability judgments, predict student performance, or imagine examinees who are just barely in a performance level. We describe applications of ID Matching in two educational testing programs and provide evidence of the effectiveness of this method. The entire process is described in the first section of the paper. Subsequent sections describe applications of ID Matching for two operational testing programs. (Contains 4 figures, 4 tables and 8 footnotes.) (As Provided).
Anmerkungen	Association of Test Publishers. 601 Pennsylvania Avenue NW, South Building, Suite 900, Washington, DC 20004. Tel: 866-240-7909; Fax: 717-755-8962; e-mail: atp@att.net; Web site: http://www.testpublishers.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10