Why I Dressed up as a Replication Bully for Halloween and Otherwise Dress as a Clown for Open Science

by Jon Grahe

Do you follow me on Twitter, are we Facebook friends? If so, you might know that my pictures almost always include me in a COS t-shirt. That is unless I’m promoting Little Jerry’s, my local burgers and breakfast joint that I also provide free endorsements for. This past summer, I traveled to Psi Chi and Council for Undergraduate Research meetings plus a family visit for three weeks. I only had 6 shirts in my carryon luggage, and there were four colors of COS t-shirts. I still wear my COS t-shirts at least weekly, and on any occasion where a new audience may have a chance to ask me about COS and various aspects of open science, and I try to make sure I post a picture of it on Facebook and Twitter. In the scientific revolution, I like to think of myself as a bard; but my colleagues might be more likely to label me as a clown.

My mother-in-law is starting to think that something is wrong with me. What do I get out of it? Maybe you are starting to wonder too; more likely you don’t really care. However, I think you should, and I welcome competition from more popular, more creative people to take the title as “most variously photographed COS t-shirt wearer” in 2015. I think the challengers should also identify a better title for the winner. So I challenge you to become more active, personal advertisers of the COS specifically and open science generally.

Reasons to Highlight the COS

The COS is about a movement, not a person or a product; though they have products I often try to sell. More accurately, I should describe their products as give-aways. Their primary give-away, the OSF website (which makes scientific workflow easier and more transparent or kept private) and is ever improving. Their other products (Open Science Badges, opportunities to contribute to meta-science projects, the OSC Blog) are similarly valuable and worth investigating. Throughout the fury about whether replication science is valuable or not, I predict that the COS will maintain a focus on methods of improving science and encouraging the positive revolution of science. I like the vision of a Scientific Utopia posited by Brian Nosek and colleagues. What role will you play in achieving that phase of our science?

Conversations From T-shirts

When considering between your COS t-shirt or other clothing options, or when considering what to talk about with your friends, family, and social network, consider the following. Social promotion of the COS benefits other sciences more than psychology. Although I have introduced some peers and students to the COS by bringing them to the COS booth at conferences last year, I think my local market is saturated. Someone should study the social psychology population about their understanding of the scientific revolution and the practices that are being employed. So much of it is happening around us. However, it is other scientists, administrators, and young people in the general public that I most frequently engage in conversation about the t-shirts and why I wear them all the time.

In these brief conversations, I talk about Psychology’s Crisis of Confidence and the importance and potential of Replication Science. I tell them about Ioannidis and that the problems extend to anywhere there is publication bias and entrenched methodology and status quo. I explain solutions offered by Open Science and the COS. My own personal agenda also includes a monologue about the potential contribution of undergraduates, to improving scientific discourse. Depending on the audience and the conversation, I may also talk about the dark side of open science. I explain my frustration with the status quo and the resistance I have witnessed to Replication Science. If they get interested, I suggest they look for phrases like “replication bully” or “replication science” and review the public discourse of the leaders in psychology. I tell them that while bullying is a potential problem, I think that the claim is inverted in this case. I invite them to draw their own conclusions and I encourage them to embrace open science where they can.

Most recently, I got a miniscule amount of Twitter activity wearing one of my COS t-shirts in a costume as a Replication Bully. I loved the costume. I included Open Science Badges as facepaint and drew them on my shield. I fashioned a nasty staff and joined my family medieval fantasy theme. Throughout the night, I probably explained that costume 25 times to peers at one event and then a host of younger people at another. Only a few prior students understood.

Many party goers ignored it completely, and some asked questions. On Twitter, I made some pithy statements about open science and the evils of the Replication Bully. What I hope is that some of the witnesses go visit the COS or talk about Open Science with their peers. Though this conversation is nauseating with each other, students and the general public seem to be interested and can help reach across scientific disciplines. Consider the disciplines impacted at your host institutions.

Halloween demands villains, and I think the Replication Bully is a villain. Bullying should be avoided, and the replication bully should not be welcome in open science circles. However, as a caricature of this discussion which still continues, it can help identify what should be avoided. Dr. Schnall suggests some parameters that should be avoided, and there are likely others. I therefore invite conversation when I wear my t-shirts, and allow the listener to draw conclusions. I want to keep the replication bully away and identifying these characteristics that are not tolerable is critical to Scientific Utopia. I explain how mistakes can be made and were made, and how they could and should be fixed. I have no shortage of opinions. At the core is the need to focus on open science principles. My experience with the COS sponsored crowd-sourcing science projects is that they always strive to achieve these principles.

Is the COS a valuable contribution to our field? Go visit their website and follow their work. There is no hurry, unless you want to miss out on an exciting part of the scientific revolution. The Reproducibility Project for all its limitations due to wide scope and challenging feasibility was a thoughtful precursor to the various Many Labs Projects. As I edit the undergraduate version of crowdsourcing science in psychology (see the Collaborative Replications and Education Project), I follow their lead. I try to maintain a similar level of quality, but it isn’t easy. They were nimble until just before data collection began, and responded to feedback and answer questions. They are also well-resourced both in their crowd of collaborators, and their collaboration with the COS. The excitement that I felt while working on the methodology summaries this summer as they prepared the pre-registered report was positive and refreshing and inviting. I wished I could do more, but there are only so many hours in a day. As I look to the future, I expect these projects will only continue to diversify. I hope they continue to answer interesting and important questions in our field.

But I think that discussion is critical and if there are criticisms of the Many Labs 1, 2, or 3 methodology, it should be reported. I predict that it will likely be addressed in Many Labs 4, 5, ..., k. I welcome the Many Labs teams into my lab, and my classroom, and I welcome their vibrant introduction of openness and persistence into our science. For the CREP, the study selection resulted from identifying studies based on impact factor and feasibility for undergraduates to complete, but we recently started soliciting requests for favorite or classic studies. How does study selection impact bias of effect estimates? I bet some of those replication science hordes (a revolutionary population filled with normally distributed personalities and interpersonal styles no doubt) will identify methods of testing this and suggesting further tests and corrections.

When I planned my Halloween costume, I thought about dressing as a hero and imitating some replication science icons, but it is a time for horror so I chose a villain. Thanksgiving is coming; this blog is not likely to be live by then, but if I have the opportunity, I’ll pose with a COS tshirt and try to represent Thanks. I’ll tweet it and post it on FB. I’ll hope that it is ingenious enough to peek some interest in the COS and meta-science projects generally and the CREP indirectly. My next opportunity is the holiday season, and then conference season will ensue. In each case, I will expose as many people as possible to open science, either through personal connection or via social media. My goal will be trying to visually connect my renewed excitement in our field with the organization that will most likely bring about the improvement many of us have craved for decades at least.

Why not embrace replication science opportunities? I saturate my social network with images of Open Science and invite them to ponder that question. While I am not a social media star, some of you might be. Pose in your COS tshirt, show the love, increase the impact and help brighten rather than darken our sciences’ future. I would like to see more interesting poses and locations than my limited imagination brings.

If you prefer to demonstrate your effort by working hard and quietly advancing open science principles, so be it. If you engage in replication bullying, I will distance myself from you. If you are fun and lively and fancy yourself a bard, but might sometimes act like a clown, expand the conversation about open science beyond psychology. As with other social movements, change will come slowly and it is likely the next generation that will see the largest gains from the current revolution in methods and scientific values. In the meantime, I’ll go put my t-shirts in the laundry and think about what they may look like on a turkey.