Assessing Mode Effects of At-Home Testing without a Randomized Trial. Research Report. ETS RR-21-10

Autor/inn/en	Kim, Sooyeon; Walker, Michael
Titel	Assessing Mode Effects of At-Home Testing without a Randomized Trial. Research Report. ETS RR-21-10
Quelle	In: ETS Research Report Series, (2021), (21 Seiten)Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2330-8516
Schlagwörter	Testing; Distance Education; Comparative Analysis; Test Items; Scores; Supervision; Testing Problems; Environmental Influences; Performance; Difficulty Level; Psychometrics + Suchen Sie Ihr Suchwort? Testdurchführung; Testen; Distance study; Distance learning; Fernunterricht; Test content; Testaufgabe; Environmental influence; Umwelteinfluss; Achievement; Leistung; Schwierigkeitsgrad; Psychometry; Psychometrie
Abstract	In this investigation, we used real data to assess potential differential effects associated with taking a test in a test center (TC) versus testing at home using remote proctoring (RP). We used a pseudo-equivalent groups (PEG) approach to examine group equivalence at the item level and the total score level. If our assumption holds that the PEG approach removes between-group ability differences (as measured by the test) reasonably well, then a plausible explanation for any systematic differences in performance between TC and RP groups that remain after applying the PEG approach would be the operation of test mode effects. At the item level, we compared item difficulties estimated using the PEG approach (i.e., adjusting only for ability differences between groups) to those estimated via delta equating (i.e., adjusting for any systematic differences between groups). All tests used in this investigation showed small, nonsystematic differences, providing evidence of trivial effects associated with at-home testing. At the total score level, we linked the RP group scores to the TC group scores after adjusting for group differences using demographic covariates. We then compared the resulting RP group conversion to the original TC group conversion (the criterion in this study). The magnitude of differences between the RP conversion and the TC conversion was small, leading to the same pass/fail decision for most RP examinees. The present analyses seem to suggest little to no mode effects for the tests used in this investigation. (As Provided).
Anmerkungen	Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01