The Impact of Using Synthetically Generated Listening Stimuli on Test-Taker Performance: A Case Study with Multiple-Choice, Single-Selection Items. TOEFL® Research Reports. RR-98. ETS?RR-22-05

Autor/inn/en	Choi, Ikkyu; Zu, Jiyun
Titel	The Impact of Using Synthetically Generated Listening Stimuli on Test-Taker Performance: A Case Study with Multiple-Choice, Single-Selection Items. TOEFL® Research Reports. RR-98. ETS?RR-22-05
Quelle	In: ETS Research Report Series, (2022), (16 Seiten)Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
Schlagwörter	Test Wiseness; Multiple Choice Tests; Test Items; Difficulty Level; Listening Skills; English (Second Language); Language Tests; Second Language Learning; Item Response Theory; Speech; Assistive Technology; Test of English as a Foreign Language + Suchen Sie Ihr Suchwort? Multiple choice examinations; Multiple-choice tests, Multiple-choice examinations; Multiple-Choice-Verfahren; Test content; Testaufgabe; Schwierigkeitsgrad; English as second language; English; Second Language; Englisch als Zweitsprache; Language test; Sprachtest; Zweitsprachenerwerb; Item-Response-Theorie; Speaking; Sprechen
Abstract	Synthetically generated speech (SGS) has become an integral part of our oral communication in a wide variety of contexts. It can be generated instantly at a low cost and allows precise control over multiple aspects of output, all of which can be highly appealing to second language (L2) assessment developers who have traditionally relied upon human voice actors for recording audio materials. Nevertheless, SGS is not widely used in L2 assessments. One major concern in this use case lies in its potential impact on test-taker performance: Would the use of SGS (as opposed to using human voice actor recordings) change how test takers respond to an item? In this study, we investigated using SGS as stimuli for English L2 listening assessment items on test-taker performance. The data came from a pilot administration of multiple new task types and included 653 test takers' responses to two versions of the same 13 items, differing only in terms of their listening stimuli: a version using human voice actor recordings and the other version with SGS files. Multifaceted comparisons between test takers' responses across the two versions showed that the two versions elicited remarkably comparable performance. The comparability provides strong empirical evidence for the use of SGS as a viable alternative for human voice actor recordings in the immediate domain of L2 assessment as well as related domains such as learning material and research instrument development. (As Provided).
Anmerkungen	Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01