A Meta-Analysis of Reliability Coefficients in Second Language Research

Ensuring internal validity in quantitative research requires, among other conditions, reliable instrumentation. Unfortunately, however, second language (L2) researchers often fail to report and even more often fail to interpret reliability estimates beyond generic benchmarks for acceptability. As a...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:The Modern language journal (Boulder, Colo.) Jg. 100; H. 2; S. 538 - 553
Hauptverfasser: PLONSKY, LUKE, DERRICK, DEIRDRE J.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Malden Blackwell Publishing Ltd 01.06.2016
Wiley Subscription Services, Inc
Wiley-Blackwell
Schlagworte:
Acceptability
Benchmarking
Coefficients
Competence
Foreign language instruction
instrumentation
Internal validity
Interrater Reliability
Language
Language acquisition
Language Proficiency
Language Research
Language teachers
Linguistics
Meta Analysis
Nonnative languages
psychometrics
Quantitative analysis
Reliability
Research design
Research Methodology
Research methods
Sample Size
second language
Second language acquisition
Second Language Learning
Second Languages
Statistical Analysis
testing
ISSN:0026-7902, 1540-4781
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Ensuring internal validity in quantitative research requires, among other conditions, reliable instrumentation. Unfortunately, however, second language (L2) researchers often fail to report and even more often fail to interpret reliability estimates beyond generic benchmarks for acceptability. As a means to guide interpretations of such estimates, this article meta-analyzes reliability coefficients (internal consistency, interrater, and intrarater) as reported in published L2 research. We recorded 2,244 reliability estimates in 537 individual articles along with study (e.g., sample size) and instrument features (e.g., item formats) proposed to influence reliability. We also coded for the indices employed (e.g., alpha, KR20). The coefficients were then aggregated (i.e., meta-analyzed). The three types of reliability varied, with internal consistency as the lowest: median = .82. Interrater and intrarater estimates were substantially higher (.92 and .95, respectively). Overall estimates were also found to vary according to study and instrument features such as proficiency (low = .79, intermediate = .84, advanced = .89) and target skill (e.g., writing = .88 vs. listening = .77). We use our results to inform and encourage interpretations of reliability estimates relative to the larger field as well as to the substantive and methodological features particular to individual studies and subdomains.
Bibliographie:istex:FBFB8FC4F5DA09E7D426308D1F3A8ED69BDE1065
ArticleID:MODL12335
ark:/67375/WNG-MLXFNZ6R-5
Northern Arizona University
ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0026-7902
1540-4781
DOI:10.1111/modl.12335