Beyond “peer review”: Should our guidelines become more inclusive?

Administration 22 Comments
By Dave Munger

Scenario 1: A physicist working with laser supercooling equipment is able to cool rubidium atoms to the lowest-yet recorded temperature. She carefully describes her exciting results and posts the paper to arXiv, where she is a registered author in good standing.

For weeks, the blogosphere explains her results and discusses the implications of the findings. But it’s three months before the experiment is actually formally published in a peer-reviewed journal, largely unaltered from the original paper she submitted to arXiv. No one bothers to blog about the published paper, which is therefore never mentioned on ResearchBlogging.org.

Scenario 2: A neuroscientist hypes a hastily-concocted brain-scan study of three people’s reactions to Britney Spears’ latest single. The “Britney neuron” is prominently covered on CNN, USA Today, and the Sun. Several bloggers link to the media accounts but offer no additional analysis, and a few bloggers make annoyed one-off posts about how the media overhypes this “science by press release.” Six months later, the research is finally presented at a neuroscience conference, but it turns out that the “Britney neuron” is also activated by the music of Bach, Bruce Springsteen, and the Jonas Brothers. It’s more of a “music region,” really, and the research doesn’t offer any new insight into how we perceive music. The work never makes it into a published journal.

In an ideal world, ResearchBlogging.org would include the blog posts about the supercooling, but not the ill-named Britney neuron. Our readers want to see the most thoughtful discussions of serious science, not celebrity-fueled media hype. But our current guidelines would reject both types of blog posts, since neither actually discusses peer-reviewed research. Although arXiv is a highly-respected resource among many of the disciplines that use it, it’s not peer-reviewed in the traditional sense.

I mentioned the possibility of opening up ResearchBlogging.org to arXiv on Twitter, and the discussion quickly took hold on FriendFeed. Here are some highlights:

When I first saw your question I thought: if this was any other repository in any other discipline I would say No, but arXiv has heft and has earned the trust by the people in the disciplines that contribute there. – Bora Zivkovic

I think if you say yes to Arxiv you will struggle to say no to eg nature precedings. I appreciate that it is different but that is not down to a clear principle but a community feeling. Which makes it very hard to base a rule on – Cameron Neylon

ResearchBlogging has the potential to become something like a syndication service for science news .. by including pre-prints alongside peer-reviewed you would start to blur the boundaries. But why not create a different section for pre-prints that track ArXiv , Nature Precedings and any other relevant ones? Pedro Beltrao

There are some parts of arxiv that are worth including, and some that aren’t, too. – Mr. Gunn

As I’ve said before, I think you really need to do this if you want participation from the physics community, particularly the theoretical high energy crowd. For them, posting to the arxiv is more or less equivalent to publication, and that’s when the interesting discussion and debate occurs. By the time some of these papers appear in a journal, they’re considered old news, and no longer worth talking about. – Chad Orzel

So there’s some enthusiastic support, some concern about distinguishing preprints from peer-reviewed research, and some concern that our overall mission will be diluted.

If we did attempt to include preprints in some way, what would our guidelines look like? Here’s what they say now about peer-review:

While there is no hard-and-fast definition of “peer-review,” peer reviewed research should meet the following guidelines:

  • Reviewed by experts in field
  • Edited
  • Archived
  • Published with clearly stated publication standards
  • Viewed as trustworthy by experts in field

I don’t think there’s any way to change that definition to include things like arXiv and exclude “science by press release.” We might be able to modify our first guideline, which says “The ‘Blogging on Peer-Reviewed Research’ icons are to be used solely to denote individual blog posts about peer-reviewed research.” We could say the icons were to be used to denote either posts about peer-reviewed research or research collected in an archive that meets the standards determined individually by discipline.

So, for example, physicists could decide that they accept research in arXiv, but biologists might decide not to accept research in Nature’s preprint archive (or the quantitative biology papers found in arXiv, for that matter).

As Chad points out, if we don’t address this problem in some way, we run the risk of never having substantial discussions about many disciplines on our site. I’m not thrilled about the idea of a separate icon / section for preprints — I think that would just make the site more difficult to use and more confusing for readers. What do you think? Is it possible to modify our guidelines in a way that includes the good stuff but still excludes the stuff we don’t like? Are there any other preprint archives that we might also want to include? Should we start slowly (perhaps just with physics and arXiv) and see how it works? Let us know in the comments — or just continue discussing the matter over on FriendFeed.

22 Responses to “Beyond “peer review”: Should our guidelines become more inclusive?”

  1. suggestion Says:
    March 28th, 2009 at 1:21 pm

    You could add to your guidelines by adapting the rough criterion already used by arXiv in its more successful areas: that the author has a conventional publication record, or is viewed as trustworthy by experts in the field.
    (”career review” as opposed to “peer review”)

  2. Olaf Davis Says:
    March 31st, 2009 at 11:44 am

    What about scenario 1a: when the paper is released on the arXiv it’s already been accepted into a respectable journal but not yet published. The gap might be less than three months, but still bloggers are unlikely to wait for it to ‘officially’ appear before writing about it. Even if arXiv prints in general aren’t allowed, these would seem like a fairly safe addition.

    (I’m assuming here that they aren’t already accepted since the guidelines say ‘published’ – but I suppose you could interpret that more broadly and say that accepted for publication in a journal + published on the arXiv = published for these standards. Is that the intention?)

  3. Черноморец Says:
    April 28th, 2009 at 4:29 pm

    Извините, как можно добавить свой материал на сайт?

  4. Kurt Says:
    May 5th, 2009 at 7:02 pm

    Maybe you could have two different icons, one for posts about peer-reviewed articles, and one for these other types of articles. Perhaps you could add a field to your database to indicate what category an article is in, and provide some mechanism for a blogger to ‘upgrade’ a post if/when a preprint is posted.

    You might also want to take a look at this post about publishing articles in the field of theoretical computer science.

  5. Kurt Says:
    May 5th, 2009 at 7:03 pm

    Um, that should have been, “if/when a preprint is *published*.”

  6. Kurt Says:
    May 5th, 2009 at 7:28 pm

    *Ahem* I guess I should have read your post a little more slowly before commenting–I see that you mention not wanting to go to a two-icon system.

    This discussion has been ongoing in one form or another since the early days of ResearchBlogging, and I have to say that I don’t completely understand your concerns about widening the scope of the articles covered. What exactly is the mission of this site? Experts in a given field will be able to judge for themselves whether a source is reliable or not. Where this site provides real benefit is for the non-expert. If there is some science article that is generating buzz, and a journalist or layman or scientist in a different field wants to learn more about it, what matters to them is the reliability of the blogger, not necessarily the original article. If the original research is not sound or not ready for publication, it can be important for that message to get out to the public, too.

  7. htpc asrock h67 Says:
    December 20th, 2011 at 12:51 am

    100% Mulberry Silk Duvets,Silk Linen Sheets,Silk Pillowcases:Revesilk

  8. Maris Mitchiner Says:
    February 7th, 2012 at 1:51 pm

    I’m pleased I found this weblog, I couldnt discover any information on this topic matter prior to. I also run a internet site and when you need to ever significant in a little bit of guest writing for me if feasible really feel zero cost to let me know, i’m generally seem for consumers to test out my internet site. Please quit by and leave a comment sometime!

  9. Boris Says:
    February 18th, 2012 at 8:34 pm

    Forum english online – http://www.english-forum.info. Study English. Learn English. English forum for begginers. Communicating on forum. Learn to understand English, talking on the forum. Self-study. Download free tutorial in English.

  10. noclegi Says:
    March 15th, 2012 at 11:05 am

    Hello! Good content material! Iím a daily website customer (just like addict ) from the website regrettably I’d a difficulty. I’m just not completely sure if it’s the right website in order to question, however, you have no spam comments. I recieve comments frequently. May a person aid me personally? Many thanks!

  11. Борис Says:
    March 20th, 2012 at 12:40 am

    Форум рыбаков на http://www.rybalkaforum.ru. Рыбалка для рыбаков. На форуме есть раздел для охотников и грибников.

  12. Сергей Says:
    June 5th, 2012 at 10:28 pm

    Продажа недвижимости в ПМР, доска объявлений по ПМР

  13. Ольга Says:
    April 1st, 2013 at 9:15 pm

    Интернет-магазин входных и межкомнатных дверей. Установка дверных доборов, подготовка проема, межкомнатных дверей, штукатурные работы. Сайт http://www.gramas.ru. Отделка откосов дверей, декоративная отделка проемов.

  14. Вячеслав Says:
    May 18th, 2013 at 9:11 pm

    Internet сайт http://www.козырек-навес.рф. Покупка навесов Форпост в розницу со склада в Калининграде. Доставка, монтаж, контактная информация. Возможность заказать (купить) козырек (навес) через отправку заказа прямо на сайте козырек-навес.рф

  15. Петр Says:
    July 23rd, 2013 at 11:13 pm

    Internet ресурс http://www.xudkartini.ru о художниках, оцените красивейшие картины мира. Биографии художников, виртуальная энциклопедия живописи. Полная энциклопедия живописи. Все об изобразительном искусстве – картины, художники, галереи. Художники готики, маньеризма, рококо, классицизма, романтизма, импрессионизма, экспрессионизма, исторического жанра, фэнтези арта.

  16. Вадим Says:
    July 28th, 2013 at 7:48 pm

    Портал http://www.fortpedagog.ru. Обучение игре на фортепиано. Интенсивный метод обучения игре на фортепиано, пианино. Методика обучения игры на фортепиано онлайн. Музыкальный опыт учащихся. Задачи обучения будущего пианиста.

  17. Вадим Says:
    July 29th, 2013 at 8:00 am

    Internet блог http://www.bolispina.ru. Лечим спину самостоятельно. Самостоятельныеметоды и способы лечения больной спины. Практические советы и рекомендации. На сайте bolispina.ru собраны древнейшие методы лечения и профилактики заболеваний спины. Китайский массаж ушу.

  18. Максим Says:
    October 17th, 2013 at 7:22 am

    Интернет блог о заработке. Информация на http://www.maximyshka2013.wix.com/maxim. Работа на дому с Интернетом для всех! Сайт научит как заработать, не вкладывая денег, просто потратив много времени. Информация о кошельках.

  19. Лена Says:
    February 5th, 2014 at 6:06 am

    Internet форум Адвентистов. Форум http://www.forum-hristian.ru. Общение о Боге. Диалоги атеистов и верующих. Найди себя.. Посты и праздники, имена и именины, иконы и жития, тропари и кондаки. Христианские знакомства. Все Христиане Здесь! Заходите на форум о христианстве, молитве, церкви, христианском прославлении, религии.

  20. Степанида Says:
    February 11th, 2014 at 5:56 am

    Портал любящих жен на http://www.receptforum.ru. На форуме делимся своими самыми любимыми рецептами приготовления вкусной и здоровой пищи. Рецепты салатов, тортов, супов, пиццы, коктейлей, домашней выпечки, первых и вторых блюд, закусок.

  21. waterproof case certificate Says:
    November 28th, 2014 at 1:45 pm

    Thanks to my father who informed me concerning
    this blog, this web site is in fact awesome.

  22. Arlen Says:
    August 24th, 2016 at 1:32 pm

    A year from now, when the expense of guaranteeing these securitiations endds up including a few
    portion points to Italy’s deficit spending, we’ll bee interested to see if the
    European Commission will be in a forgiving state of mind given that they approved
    this brand-new plan.

Leave a Reply

Icons by N.Design Studio. Based on a theme by Ben Swift.
Entries RSS Comments RSS Log in