View in EDS

What feedback do reviewers give when reviewing qualitative manuscripts? A focused mapping review and synthesis

Saved in:
Bibliographic Details
Title: What feedback do reviewers give when reviewing qualitative manuscripts? A focused mapping review and synthesis
Authors: Herber, Oliver Rudolf, Overig, Bradbury-Jones, Caroline, Overig, Böling, Susanna, Overig, Combes, Sarah, Overig, Hirt, Julian, Overig, Koop, Yvonne, Overig, Nyhagen, Ragnhild, Overig, Veldhuizen, Jessica, Lid lectoraat, Taylor, Julie, Overig
Contributors: Chronisch Zieken, Hogeschool Utrecht@@@Kenniscentrum Gezond en Duurzaam Leven
Source: BMC Medical Research Methodology. (20):122
Publisher Information: Hogeschool Utrecht, 2020.
Publication Year: 2020
Subject Terms: open access publishing, peer review, manuscript review, qualitative analysis, qualitative research, synthesis, mapping
Description: Background: Peer review is at the heart of the scientific process. With the advent of digitisation, journals started to offer electronic articles or publishing online only. A new philosophy regarding the peer review process found its way into academia: the open peer review. Open peer review as practiced by BioMed Central (BMC) is a type of peer review where the names of authors and reviewers are disclosed and reviewer comments are published alongside the article. A number of articles have been published to assess peer reviews using quantitative research. However, no studies exist that used qualitative methods to analyse the content of reviewers’ comments. Methods: A focused mapping review and synthesis (FMRS) was undertaken of manuscripts reporting qualitative research submitted to BMC open access journals from 1 January – 31 March 2018. Free-text reviewer comments were extracted from peer review reports using a 77-item classification system organised according to three key dimensions that represented common themes and sub-themes. A two stage analysis process was employed. First, frequency counts were undertaken that allowed revealing patterns across themes/sub-themes. Second, thematic analysis was conducted on selected themes of the narrative portion of reviewer reports. Results: A total of 107 manuscripts submitted to nine open-access journals were included in the FMRS. The frequency analysis revealed that among the 30 most frequently employed themes 'writing criteria' (dimension II) is the top ranking theme, followed by comments in relation to the 'methods' (dimension I). Besides that, some results suggest an underlying quantitative mindset of reviewers. Results are compared and contrasted in relation to established reporting guidelines for qualitative research to inform reviewers and authors of frequent feedback offered to enhance the quality of manuscripts. Conclusions: This FMRS has highlighted some important issues that hold lessons for authors, reviewers and editors. We suggest modifying the current reporting guidelines by including a further item called 'Degree of data transformation' to prompt authors and reviewers to make a judgment about the appropriateness of the degree of data transformation in relation to the chosen analysis method. Besides, we suggest that completion of a reporting checklist on submission becomes a requirement.
Document Type: article
Language: English
DOI: 10.1186/s12874-020-01005-y
Access URL: https://surfsharekit.nl/public/d4f833b1-01bb-4fbc-9f0c-9818b18831c3
https://surfsharekit.nl/objectstore/58499a2a-698a-4444-b821-efe78cd4ca4b
Availability: http://www.hbo-kennisbank.nl/en/page/hborecord.view/?uploadId=sharekit_hu:oai:surfsharekit.nl:d4f833b1-01bb-4fbc-9f0c-9818b18831c3
Accession Number: edshbo.sharekit.hu.oai.surfsharekit.nl.d4f833b1.01bb.4fbc.9f0c.9818b18831c3
Database: HBO Kennisbank
Description
Abstract:Background: Peer review is at the heart of the scientific process. With the advent of digitisation, journals started to offer electronic articles or publishing online only. A new philosophy regarding the peer review process found its way into academia: the open peer review. Open peer review as practiced by BioMed Central (BMC) is a type of peer review where the names of authors and reviewers are disclosed and reviewer comments are published alongside the article. A number of articles have been published to assess peer reviews using quantitative research. However, no studies exist that used qualitative methods to analyse the content of reviewers’ comments. Methods: A focused mapping review and synthesis (FMRS) was undertaken of manuscripts reporting qualitative research submitted to BMC open access journals from 1 January – 31 March 2018. Free-text reviewer comments were extracted from peer review reports using a 77-item classification system organised according to three key dimensions that represented common themes and sub-themes. A two stage analysis process was employed. First, frequency counts were undertaken that allowed revealing patterns across themes/sub-themes. Second, thematic analysis was conducted on selected themes of the narrative portion of reviewer reports. Results: A total of 107 manuscripts submitted to nine open-access journals were included in the FMRS. The frequency analysis revealed that among the 30 most frequently employed themes 'writing criteria' (dimension II) is the top ranking theme, followed by comments in relation to the 'methods' (dimension I). Besides that, some results suggest an underlying quantitative mindset of reviewers. Results are compared and contrasted in relation to established reporting guidelines for qualitative research to inform reviewers and authors of frequent feedback offered to enhance the quality of manuscripts. Conclusions: This FMRS has highlighted some important issues that hold lessons for authors, reviewers and editors. We suggest modifying the current reporting guidelines by including a further item called 'Degree of data transformation' to prompt authors and reviewers to make a judgment about the appropriateness of the degree of data transformation in relation to the chosen analysis method. Besides, we suggest that completion of a reporting checklist on submission becomes a requirement.
DOI:10.1186/s12874-020-01005-y