Literaturnachweis - Detailanzeige
| Autor/in | Olney, Andrew M. |
|---|---|
| Titel | Generating Multiple Choice Questions with a Multi-Angle Question Answering Model [Konferenzbericht] Paper presented at the Workshop of the Learner Data Institute (3rd, Durham, UK, Jul 27, 2022). |
| Quelle | (2022), (7 Seiten)
PDF als Volltext (1); PDF als Volltext kostenfreie Datei (2) |
| Zusatzinformation | ORCID (Olney, Andrew M.) Weitere Informationen |
| Sprache | englisch |
| Dokumenttyp | gedruckt; online; Monographie |
| Schlagwörter | Test Construction; Multiple Choice Tests; Test Items; Models; Textbooks; Natural Language Processing |
| Abstract | Multi-angle question answering models have recently been proposed that promise to perform related tasks like question generation. However, performance on related tasks has not been thoroughly studied. We investigate a leading model called Macaw on the task of multiple choice question generation and evaluate its performance on three angles that systematically reduce the complexity of the task. Our results indicate that despite the promise of generalization, Macaw performs poorly on untrained angles. Even on a trained angle, Macaw fails to generate four distinct multiple-choice options on 17% of inputs. We propose augmenting multiple-choice options by paraphrasing angle input and show this increases overall success to 97.5%. A human evaluation comparing the augmented multiple-choice questions with textbook questions on the same topic reveals that Macaw questions broadly score highly but below human questions. [This paper was published in: "Proceedings of the 3rd Workshop of the Learner Data Institute," edited by S. E. Fancsali and V. Rus, Learner Data Institute, 2022, pp. 18-23.] (As Provided). |
| Erfasst von | ERIC (Education Resources Information Center), Washington, DC |
| Update | 2024/1/01 |
Literaturbeschaffung und Bestandsnachweise in Bibliotheken prüfen
Permalink als QR-Code
Inhalt auf sozialen Plattformen teilen (nur vorhanden, wenn Javascript eingeschaltet ist)
Teile diese Seite:
