Evaluating the Effectiveness of the Training Program on Direct and Semi-Direct Oral Proficiency Assessment: A Case of Multifaceted Rasch Analysis

Autor/in	Bijani, Houman
Titel	Evaluating the Effectiveness of the Training Program on Direct and Semi-Direct Oral Proficiency Assessment: A Case of Multifaceted Rasch Analysis
Quelle	In: Cogent Education, 6 (2019) 1, Artikel 1670592 (20 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2331-186X
DOI	10.1080/2331186X.2019.1670592
Schlagwörter	Program Effectiveness; Oral Language; Language Proficiency; Language Tests; Pretests Posttests; Interviews; English (Second Language); Second Language Learning; Second Language Instruction; Task Analysis; Teaching Methods; Interrater Reliability; Scores; Stress Variables; Difficulty Level; Language Teachers; Teacher Attitudes; Faculty Development; Adult Students; Foreign Countries; Iran + Suchen Sie Ihr Suchwort? Oral interpretation; Mündlicher Sprachgebrauch; Language skill; Language skills; Sprachkompetenz; Language test; Sprachtest; Interviewing; Interviewtechnik; English as second language; English; Second Language; Englisch als Zweitsprache; Zweitsprachenerwerb; Fremdsprachenunterricht; Aufgabenanalyse; Teaching method; Lehrmethode; Unterrichtsmethode; Interrater-Reliabilität; Schwierigkeitsgrad; Language teacher; Sprachunterricht; Lehrerverhalten; Adult; Adults; Student; Students; Erwachsenenalter; Studentin; Schüler; Schülerin; Ausland
Abstract	An Oral Proficiency Interview (OPI) may be evaluated either during the interview procedure (direct-method) or from a tape-made oral interaction (semi-direct method). Such variety in methods of assessment can influence test takers' scores extensively. However, it is not conclusive whether such differences are due to test format or something else. Besides, most previous studies have investigated the application of multifaceted Rasch measurement (MFRM) on only one or two facets and few studies have used a pre- and post-training design. 20 English as a foreign language (EFL) teachers rated the oral performances produced by 200 test takers before and after a training program. The tasks were implemented via two methods of task delivery, direct and semi-direct. The findings indicated that test takers found semi-direct oral tests harder and more stressful than direct oral tests since they are more difficult due to having more complicated linguistic and communicative features and contained more lexically complex structures. Therefore, direct oral tests are more appropriate for low-level test takers, and semi-direct tests for higher ability ones. Data analyses demonstrated no significant difference between the ratings of direct and semi-direct oral assessment by the raters. Consequently, semi-direct oral tests can be regarded as a reliable substitute for direct oral tests. Finally, the outcomes showed that although training can result in higher levels of interrater consistency and reduced levels of severity/leniency, biasedness and inconsistency, it cannot turn rater into duplicates of one another. (As Provided).
Anmerkungen	Cogent OA. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01