Comparing Holistic and Analytic Marking Methods in Assessing Speech Act Production in L2 Chinese

Autor/inn/en	Li, Shuai; Wen, Ting; Li, Xian; Feng, Yali; Lin, Chuan
Titel	Comparing Holistic and Analytic Marking Methods in Assessing Speech Act Production in L2 Chinese
Quelle	In: Language Testing, 40 (2023) 2, S.249-275 (27 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Li, Shuai)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0265-5322
DOI	10.1177/02655322221113917
Schlagwörter	Speech Acts; Second Language Learning; Second Language Instruction; Chinese; Evaluators; Scoring; Holistic Approach; Test Items; North Americans; Oral Language; Language Tests; Pragmatics; Native Language; Rating Scales; Language Fluency; Intonation; Suprasegmentals; Correlation; Grammar; Item Analysis; Study Abroad; Foreign Students; Foreign Countries; China + Suchen Sie Ihr Suchwort? Sprechakt; Zweitsprachenerwerb; Fremdsprachenunterricht; China; Chinesen; Bewertung; Holistischer Ansatz; Test content; Testaufgabe; Oral interpretation; Mündlicher Sprachgebrauch; Language test; Sprachtest; Pragmalinguistik; Rating-Skala; Language skill; Language skills; Sprachkompetenz; Korrelation; Grammatik; Itemanalyse; Studies abroad; Auslandsstudium; Ausland
Abstract	This study compared holistic and analytic marking methods for their effects on parameter estimation (of examinees, raters, and items) and rater cognition in assessing speech act production in L2 Chinese. Seventy American learners of Chinese completed an oral Discourse Completion Test assessing requests and refusals. Four first-language (L1) Chinese raters evaluated the examinees' oral productions using two four-point rating scales. The holistic scale simultaneously included the following five dimensions: communicative function, prosody, fluency, appropriateness, and grammaticality; the analytic scale included sub-scales to examine each of the five dimensions. The raters scored the dataset twice with the two marking methods, respectively, and with counterbalanced order. They also verbalized their scoring rationale after performing each rating. Results revealed that both marking methods led to high reliability and produced scores with high correlation; however, analytic marking possessed better assessment quality in terms of higher reliability and measurement precision, higher percentages of Rasch model fit for examinees and items, and more balanced reference to rating criteria among raters during the scoring process. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01