Human versus Computer Partner in the Paired Oral Discussion Test

Autor/inn/en	Ockey, Gary J.; Chukharev-Hudilainen, Evgeny
Titel	Human versus Computer Partner in the Paired Oral Discussion Test
Quelle	In: Applied Linguistics, 42 (2021) 5, S.924-944 (21 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0142-6001
DOI	10.1093/applin/amaa067
Schlagwörter	Oral Language; Grammar; Language Fluency; Language Tests; Communicative Competence (Languages); Evaluators; Task Analysis; Computer Assisted Testing; Artificial Intelligence; Interpersonal Communication; Scores; Comparative Analysis; Standardized Tests; Discussion; Discourse Analysis + Suchen Sie Ihr Suchwort? Oral interpretation; Mündlicher Sprachgebrauch; Grammatik; Language skill; Language skills; Sprachkompetenz; Language test; Sprachtest; Communicative competence; Languages; Kommunikative Kompetenz; Sprache; Aufgabenanalyse; Künstliche Intelligenz; Interpersonale Kommunikation; Standadised tests; Standardisierter Test; Diskussion; Diskursanalyse
Abstract	A challenge of large-scale oral communication assessments is to feasibly assess a broad construct that includes interactional competence. One possible approach in addressing this challenge is to use a spoken dialog system (SDS), with the computer acting as a peer to elicit a ratable speech sample. With this aim, an SDS was built and four trained human raters assessed the discourse elicited from 40 test takers that completed a paired oral task with both a human and a computer partner. The test takers were evaluated based on the analytic operational oral communication rating scales which included interactional competence, fluency, pronunciation, and grammar/vocabulary. Repeated-measures ANOVA indicated that fluency, pronunciation, and grammar and vocabulary were scored similarly across the two conditions, while interactional competence was scored substantially higher in the human partner condition. A g-study indicated that the computer partner was more reliable in assessing interactional competence, and rater questionnaire and interview data suggested the computer provided a more standardized assessment. Conversely, raters generally favored the human partner, in part because of its perceived authenticity and naturalness. (As Provided).
Anmerkungen	Oxford University Press. Great Clarendon Street, Oxford, OX2 6DP, UK. Tel: +44-1865-353907; Fax: +44-1865-353485; e-mail: jnls.cust.serv@oxfordjournals.org; Web site: http://applij.oxfordjournals.org/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01