Improving the reporting quality of reliability generalization meta-analyses: The REGEMA checklist

Reliability generalization (RG) is a meta-analytic approach that aims to characterize how reliability estimates from the same test vary across different applications of the instrument. With this purpose RG meta-analyses typically focus on a particular test and intend to obtain an overall reliability...

Full description

Saved in:
Bibliographic Details
Published in:Research synthesis methods Vol. 12; no. 4; p. 516
Main Authors: Sánchez-Meca, Julio, Marín-Martínez, Fulgencio, López-López, José Antonio, Núñez-Núñez, Rosa Maria, Rubio-Aparicio, María, López-García, Juan José, López-Pina, José Antonio, Blázquez-Rincón, Desirée Mª, López-Ibáñez, Carmen, López-Nicolás, Rubén
Format: Journal Article
Language:English
Published: England 01.07.2021
Subjects:
reporting quality
meta-analysis
reliability generalization
reliability coefficient
systematic reviews
ISSN:1759-2887, 1759-2887
Online Access:Get more information
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Reliability generalization (RG) is a meta-analytic approach that aims to characterize how reliability estimates from the same test vary across different applications of the instrument. With this purpose RG meta-analyses typically focus on a particular test and intend to obtain an overall reliability of test scores and to investigate how the composition and variability of the samples affect reliability. Although several guidelines have been proposed in the meta-analytic literature to help authors improve the reporting quality of meta-analyses, none of them were devised for RG meta-analyses. The purpose of this investigation was to develop REGEMA (REliability GEneralization Meta-Analysis), a 30-item checklist (plus a flow chart) adapted to the specific issues that the reporting of an RG meta-analysis must take into account. Based on previous checklists and guidelines proposed in the meta-analytic arena, a first version was elaborated by applying the nominal group methodology. The resulting instrument was submitted to a list of independent meta-analysis experts and, after discussion, the final version of the REGEMA checklist was reached. In a pilot study, four pairs of coders applied REGEMA to a random sample of 40 RG meta-analyses in Psychology, and results showed satisfactory inter-coder reliability. REGEMA can be used by: (a) meta-analysts conducting or reporting an RG meta-analysis and aiming to improve its reporting quality; (b) consumers of RG meta-analyses who want to make informed critical appraisals of their reporting quality, and (c) reviewers and editors of journals who are considering submissions where an RG meta-analysis was reported for potential publication.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:1759-2887
1759-2887
DOI:10.1002/jrsm.1487