Writing Quality Predictive Modeling: Integrating Register-Related Factors

Autor/inn/en	Wang, Heqiao; Troia, Gary A.
Titel	Writing Quality Predictive Modeling: Integrating Register-Related Factors
Quelle	In: Written Communication, 40 (2023) 4, S.1070-1112 (43 Seiten) PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Wang, Heqiao) ORCID (Troia, Gary A.) Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0741-0883
DOI	10.1177/07410883231185287
Schlagwörter	Writing Evaluation; Essays; Elementary School Students; Grade 4; Grade 5; Automation; Prediction; Models; Accuracy; Scoring; Writing Skills; Computational Linguistics + Suchen Sie Ihr Suchwort? Essay; Aufsatzunterricht; School year 04; 4. Schuljahr; Schuljahr 04; School year 05; 5. Schuljahr; Schuljahr 05; Vorhersage; Analogiemodell; Bewertung; Writing skill; Schreibfertigkeit; Linguistics; Computerlinguistik
Abstract	The primary purpose of this study is to investigate the degree to which register knowledge, register-specific motivation, and diverse linguistic features are predictive of human judgment of writing quality in three registers--narrative, informative, and opinion. The secondary purpose is to compare the evaluation metrics of register-partitioned automated writing evaluation models in three conditions: (1) register-related factors alone, (2) linguistic features alone, and (3) the combination of these two. A total of 1006 essays (n = 327, 342, and 337 for informative, narrative, and opinion, respectively) written by 92 fourth- and fifth-graders were examined. A series of hierarchical linear regression analyses controlling for the effects of demographics were conducted to select the most useful features to capture text quality, scored by humans, in the three registers. These features were in turn entered into automated writing evaluation predictive models with tuning of the parameters in a tenfold cross-validation procedure. The average validity coefficients (i.e., quadratic-weighed kappa, Pearson correlation r, standardized mean score difference, score deviation analysis) were computed. The results demonstrate that (1) diverse feature sets are utilized to predict quality in the three registers, and (2) the combination of register-related factors and linguistic features increases the accuracy and validity of all human and automated scoring models, especially for the registers of informative and opinion writing. The findings from this study suggest that students' register knowledge and register-specific motivation add additional predictive information when evaluating writing quality across registers beyond that afforded by linguistic features of the paper itself, whether using human scoring or automated evaluation. These findings have practical implications for educational practitioners and scholars in that they can help strengthen consideration of register-specific writing skills and cognitive and motivational forces that are essential components of effective writing instruction and assessment. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01