New “Reviewer Statement” Initiative Aims to (Further) Improve Community Norms Toward Disclosure

by Etienne LeBel

An Open Science Collaboration -- made up of Uri Simonsohn, Etienne LeBel, Don Moore, Leif D. Nelson, Brian Nosek, and Joe Simmons -- is glad to announce a new initiative aiming to improve community norms toward the disclosure of basic methodological information during the peer-review process. Endorsed by the Center for Open Science, the initiative involves a standard reviewer statement that any peer reviewer can include in their review requesting that authors add a statement to the paper confirming that they have disclosed all data exclusions, experimental conditions, assessed measures, and how they determined their samples sizes (following from the 21-word solution; Simmons, Nelson, & Simonsohn, 2012, 2013; see also PsychDisclosure.org; LeBel et al., 2013). Here is the statement, which is available on the Open Science Framework:

"I request that the authors add a statement to the paper confirming whether, for all experiments, they have reported all measures, conditions, data exclusions, and how they determined their sample sizes. The authors should, of course, add any additional text to ensure the statement is accurate. This is the standard reviewer disclosure request endorsed by the Center for Open Science (see http://osf.io/project/hadz3). I include it in every review."

The idea originated from the realization that as peer reviewers, we typically lack fundamental information regarding how the data was collected and analyzed which prevents us from be able to properly evaluate the claims made in a submitted manuscript. Some reviewers interested in requesting such information, however, were concerned that such requests would make them appear selective and/or compromise their anonymity. Discussions ensued and contributors developed a standard reviewer disclosure request statement that overcomes these concerns and allows the community of reviewers to improve community norms toward the disclosure of such methodological information across all journals and articles.

Some of the contributors, including myself, were hoping for a reviewer statement with a bit more teeth. For instance, requesting the disclosure of such information as a requirement before accepting to review an article or requiring the re-review of a revised manuscript once the requested information has been disclosed. The team of contributors, however, ultimately decided that it would be better to start small to get acceptance, in order to maximize the probability that the initiative has an impact in shaping the community norms.

Hence, next time you are invited to review a manuscript for publication at any journal, please remember to include the reviewer disclosure statement!

References

LeBel, E. P., Borsboom, D., Giner-Sorolla, R., Hasselman, F., Peters, K. R., Ratliff, K. A., & Smith, C. T. (2013). PsychDisclosure.org: Grassroots support for reforming reporting standards in psychology. Perspectives on Psychological Science, 8(4), 424-432. doi: 10.1177/1745691613491437

Simmons J., Nelson L. & Simonsohn U. (2011) False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allow Presenting Anything as Significant. Psychological Science, 22(11), 1359-1366.

Simmons J., Nelson L. & Simonsohn U. (2012) A 21 Word Solution. Dialogue: The Official Newsletter of the Society for Personality and Social Psychology, 26(2), 4-7.