U.S. flag

An official website of the United States government, Department of Justice.

Locating unregistered and unreported data for use in a social science systematic review and meta-analysis.

NCJ Number
303041
Author(s)
Date Published
2020
Annotation

This study attempted to locate one type of meta-analytic data, i.e., findings from studies that neither registered nor reported the collected outcome data.

Abstract

 Meta-analysts rely on the availability of data from previously conducted studies, i.e., they rely on primary study authors to register their outcome data, either in a study’s text or on publicly available websites, and report the results of their work, either again in a study’s text or on publicly accessible data repositories. If a primary study author does not register data collection and similarly does not report the data collection results, the meta-analyst is at risk of failing to include the collected data. The current study conducted a large-scale search for potential studies and emailed an author query request to more than 600 primary study authors to ask whether they had collected eligible outcome data. Responses were received from 75 authors (12.3 percent), three of whom sent eligible findings. The results of the search confirmed the proof of concept (i.e., that authors collect data but fail to register or report it publicly), and the meta-analytic results indicated that excluding the identified studies would change some of the researchers substantive conclusions. Cost analyses indicated, however, a high price to finding the missing studies. This article concludes by reaffirming calls for greater adoption of primary study pre-registration, as well as data archiving in publicly available repositories. (publisher abstract modified)

Date Published: January 1, 2020