Open Access Protocol

A protocol for a systematic review on the impact of unpublished studies and studies published in the gray literature in meta-analyses

Christine Schmucker1, Annette Bluemle1, Matthias Briel23, Susan Portalupi1, Britta Lang1, Edith Motschall4, Guido Schwarzer4, Dirk Bassler5, Katharina F Mueller5, Erik von Elm16, Joerg J Meerpohl1* and on behalf of the OPEN Consortium

Author Affiliations

1 German Cochrane Center, Institute of Medical Biometry and Medical Informatics, University Medical Center Freiburg, 79110 Freiburg, Germany

2 Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Basel, 4031, Switzerland

3 Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada

4 Institute of Medical Biometry and Medical Informatics, University Medical Centre Freiburg, Freiburg, 79104, Germany

5 Centre for Paediatric Clinical Studies, University Medical Center Tuebingen, Tuebingen, 72070, Germany

6 Cochrane Switzerland, IUMSP, University Hospital Lausanne, Lausanne, 1005, Switzerland

For all author emails, please log on.

Systematic Reviews 2013, 2:24  doi:10.1186/2046-4053-2-24

Published: 2 May 2013

Abstract

Background

Meta-analyses are particularly vulnerable to the effects of publication bias. Despite methodologists’ best efforts to locate all evidence for a given topic the most comprehensive searches are likely to miss unpublished studies and studies that are published in the gray literature only. If the results of the missing studies differ systematically from the published ones, a meta-analysis will be biased with an inaccurate assessment of the intervention’s effects.

As part of the OPEN project (http://www.open-project.eu webcite) we will conduct a systematic review with the following objectives:

▪ To assess the impact of studies that are not published or published in the gray literature on pooled effect estimates in meta-analyses (quantitative measure).

▪ To assess whether the inclusion of unpublished studies or studies published in the gray literature leads to different conclusions in meta-analyses (qualitative measure).

Methods/Design

Inclusion criteria: Methodological research projects of a cohort of meta-analyses which compare the effect of the inclusion or exclusion of unpublished studies or studies published in the gray literature.

Literature search: To identify relevant research projects we will conduct electronic searches in Medline, Embase and The Cochrane Library; check reference lists; and contact experts.

Outcomes: 1) The extent to which the effect estimate in a meta-analyses changes with the inclusion or exclusion of studies that were not published or published in the gray literature; and 2) the extent to which the inclusion of unpublished studies impacts the meta-analyses’ conclusions.

Data collection: Information will be collected on the area of health care; the number of meta-analyses included in the methodological research project; the number of studies included in the meta-analyses; the number of study participants; the number and type of unpublished studies; studies published in the gray literature and published studies; the sources used to retrieve studies that are unpublished, published in the gray literature, or commercially published; and the validity of the methodological research project.

Data synthesis: Data synthesis will involve descriptive and statistical summaries of the findings of the included methodological research projects.

Discussion

Results are expected to be publicly available in the middle of 2013.

Keywords:
Publication bias; Gray literature; Unpublished studies; Meta-analyses; The OPEN project