Assessing the Impact of a Test Question: Evidence from the "Underground Railroad" Controversy

Autor/inn/en	Dee, Thomas S.; Domingue, Benjamin W.
Titel	Assessing the Impact of a Test Question: Evidence from the "Underground Railroad" Controversy
Quelle	In: Educational Measurement: Issues and Practice, 40 (2021) 2, S.81-88 (8 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Domingue, Benjamin W.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0731-1745
DOI	10.1111/emip.12411
Schlagwörter	High School Students; Grade 10; Language Arts; High Stakes Tests; Essay Tests; Novels; African American Literature; Test Items; Racial Bias; Test Bias; Controversial Issues (Course Content); Stereotypes; African American Students; Massachusetts + Suchen Sie Ihr Suchwort? High school; High schools; Student; Students; Oberschule; Schüler; Schülerin; Studentin; Sprachkultur; Schriftlicher Sprachgebrauch; Novel; Roman; Test content; Testaufgabe; Racial discrimination; Rassismus; Testkritik; Controversial issues; Kontroverse; Klischee; African Americans; Afroamerikaner; Master-Studiengang
Abstract	On the second day of a 2019 high-stakes English Language Arts assessment, Massachusetts 10th graders faced an essay question that was based on a passage from the novel "The Underground Railroad" and publicly characterized as racially insensitive. Though the state excluded the essay responses from student scores, an unresolved public controversy focused on whether this question created a racial bias in performance on the remaining test items. We present the results from an independent, preregistered study of this question. Our confirmatory results indicate that exposure to the controversial question is associated with a small reduction in the comparative performance of Black students on the overall test (approximately 0.006[sigma]). However, we also find a wide dispersion of such effects when examining similarly small sets of test items from prior state assessments that "lacked" a controversial question, which suggests the 2019 assessment was not distinctive. Our approach offers a potential template that may be useful in other contexts where testing controversies occur and underscores the importance of carefully screening test items to avoid such occurrences. (As Provided).
Anmerkungen	Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01