Using Subject Matter Experts To Assess Content Representation: A MDS Analysis.

Autor/inn/en	Sireci, Stephen G.; Geisinger, Kurt
Institution	American Council on Education, Washington, DC. GED Testing Service.
Titel	Using Subject Matter Experts To Assess Content Representation: A MDS Analysis.
Quelle	(1993), (66 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Achievement Tests; Certified Public Accountants; Cluster Analysis; Content Analysis; Content Validity; Correlation; Factor Analysis; Higher Education; Item Analysis; Junior High Schools; Licensing Examinations (Professions); Mathematical Models; Matrices; Multidimensional Scaling; Research Methodology; Social Studies; Standardized Tests; Test Construction; Test Items + Suchen Sie Ihr Suchwort? Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Clusteranalyse; Inhaltsanalyse; Korrelation; Faktorenanalyse; Hochschulbildung; Hochschulsystem; Hochschulwesen; Itemanalyse; Sekundarstufe I; Mathematical model; Mathematisches Modell; Matrizenrechnung; Mehrdimensionale Analysis; Research method; Forschungsmethode; Gemeinschaftskunde; Standadised tests; Standardisierter Test; Testaufbau; Test content; Testaufgabe
Abstract	Various methods used to assess the content of a test are reviewed, and a new procedure designed to improve on these methods is presented. The two tests considered are a professional licensure examination, the auditing section of the Uniform Certified Public Accountant Examination, and an educational achievement test, a nationally standardized social studies achievement test. Previous methods have generally been empirical, using factor analysis or multidimensional scaling (MDS) to analyze the inter-item correlation matrix derived from examinee responses, or subjective, using the data provided by subject matter experts (SMEs) to determine whether items represent content areas that the test purports to measure. A method has previously been proposed that uses MDS to discover dimensions obtained from the analysis of ratings by SMEs of the similarity of items comprising a test. This study expanded that method by using 2 groups of SMEs (15 for each test) to evaluate the content of the 2 tests studied. Correlation and cluster analyses results suggest that the content structure of a test can be evaluated adequately by analyzing item similarity data provided by SMEs. Results further suggest that the MDS procedure should be used to supplement analyses of item relevance data rather than replace them. Six figures and 18 tables present analysis findings. (Contains 23 references.) (SLD)
Erfasst von	ERIC (Education Resources Information Center), Washington, DC