ResearchBlogging.org News

Scenario 1: A physicist working with laser supercooling equipment is able to cool rubidium atoms to the lowest-yet recorded temperature. She carefully describes her exciting results and posts the paper to arXiv, where she is a registered author in good standing.

For weeks, the blogosphere explains her results and discusses the implications of the findings. But it’s three months before the experiment is actually formally published in a peer-reviewed journal, largely unaltered from the original paper she submitted to arXiv. No one bothers to blog about the published paper, which is therefore never mentioned on ResearchBlogging.org.

Scenario 2: A neuroscientist hypes a hastily-concocted brain-scan study of three people’s reactions to Britney Spears’ latest single. The “Britney neuron” is prominently covered on CNN, USA Today, and the Sun. Several bloggers link to the media accounts but offer no additional analysis, and a few bloggers make annoyed one-off posts about how the media overhypes this “science by press release.” Six months later, the research is finally presented at a neuroscience conference, but it turns out that the “Britney neuron” is also activated by the music of Bach, Bruce Springsteen, and the Jonas Brothers. It’s more of a “music region,” really, and the research doesn’t offer any new insight into how we perceive music. The work never makes it into a published journal.

In an ideal world, ResearchBlogging.org would include the blog posts about the supercooling, but not the ill-named Britney neuron. Our readers want to see the most thoughtful discussions of serious science, not celebrity-fueled media hype. But our current guidelines would reject both types of blog posts, since neither actually discusses peer-reviewed research. Although arXiv is a highly-respected resource among many of the disciplines that use it, it’s not peer-reviewed in the traditional sense.

I mentioned the possibility of opening up ResearchBlogging.org to arXiv on Twitter, and the discussion quickly took hold on FriendFeed. Here are some highlights:

When I first saw your question I thought: if this was any other repository in any other discipline I would say No, but arXiv has heft and has earned the trust by the people in the disciplines that contribute there. – Bora Zivkovic

I think if you say yes to Arxiv you will struggle to say no to eg nature precedings. I appreciate that it is different but that is not down to a clear principle but a community feeling. Which makes it very hard to base a rule on – Cameron Neylon

ResearchBlogging has the potential to become something like a syndication service for science news .. by including pre-prints alongside peer-reviewed you would start to blur the boundaries. But why not create a different section for pre-prints that track ArXiv , Nature Precedings and any other relevant ones? Pedro Beltrao

There are some parts of arxiv that are worth including, and some that aren’t, too. – Mr. Gunn

As I’ve said before, I think you really need to do this if you want participation from the physics community, particularly the theoretical high energy crowd. For them, posting to the arxiv is more or less equivalent to publication, and that’s when the interesting discussion and debate occurs. By the time some of these papers appear in a journal, they’re considered old news, and no longer worth talking about. – Chad Orzel

So there’s some enthusiastic support, some concern about distinguishing preprints from peer-reviewed research, and some concern that our overall mission will be diluted.

If we did attempt to include preprints in some way, what would our guidelines look like? Here’s what they say now about peer-review:

While there is no hard-and-fast definition of “peer-review,” peer reviewed research should meet the following guidelines:

• Reviewed by experts in field
• Edited
• Archived
• Published with clearly stated publication standards
• Viewed as trustworthy by experts in field

I don’t think there’s any way to change that definition to include things like arXiv and exclude “science by press release.” We might be able to modify our first guideline, which says “The ‘Blogging on Peer-Reviewed Research’ icons are to be used solely to denote individual blog posts about peer-reviewed research.” We could say the icons were to be used to denote either posts about peer-reviewed research or research collected in an archive that meets the standards determined individually by discipline.

So, for example, physicists could decide that they accept research in arXiv, but biologists might decide not to accept research in Nature’s preprint archive (or the quantitative biology papers found in arXiv, for that matter).

As Chad points out, if we don’t address this problem in some way, we run the risk of never having substantial discussions about many disciplines on our site. I’m not thrilled about the idea of a separate icon / section for preprints — I think that would just make the site more difficult to use and more confusing for readers. What do you think? Is it possible to modify our guidelines in a way that includes the good stuff but still excludes the stuff we don’t like? Are there any other preprint archives that we might also want to include? Should we start slowly (perhaps just with physics and arXiv) and see how it works? Let us know in the comments — or just continue discussing the matter over on FriendFeed.