Yesterday, October 22nd, six scientists and one government official were sentenced to six years in prison for manslaughter. A consequence of the events surrounding the earthquake (magnitude: 6.3) that hit the Italian city of L’Aquila on April 6th 2009, and … Continue reading →... Read more »
Nosengo, N. (2012) Italian court finds seismologists guilty of manslaughter. Nature. DOI: 10.1038/nature.2012.11640
How to ensure that results in psychology (and other fields) are replicated has become a popular topic of discussion recently. There's no doubt that many results fail to replicate, and also, that people don't even try to replicate findings as much as they should.Yet psychologist Gregory Francis warns that replication per se is not always a good thing: Publication bias and the failure of replication in experimental psychologyAmong experimental psychologists, successful replication enhances belief in a finding, while a failure to replicate is often interpreted to mean that one of the experiments is flawed. This view is wrong. Because experimental psychology uses statistics, empirical findings should appear with predictable probabilities. In a misguided effort to demonstrate successful replication of empirical findings and avoid failures to replicate, experimental psychologists sometimes report too many positive results.Rather than strengthen confidence in an effect, too much successful replication actually indicates publication bias, which invalidates entire sets of experimental findings...Even populations with strong effects should have some experiments that do not reject the null hypothesis. Such null findings should not be interpreted as failures to replicate, because if the experiments are run properly and reported fully, such nonsignificantfindings are an expected outcome of random sampling... If there are not enough null findings in a set of moderately powered experiments, the experiments were either not run properly or not fully reported. If experiments are not run properly or not reported fully, there is no reason to believe the reported effect is real.Say you took a pack of playing cards and removed half the red cards. Your pack would now be 2/3rds black, so if you took a random sample of cards, say a poker hand of 5 cards, then you'd expect more blacks than reds (a significant 'effect' of color). But you'd still expect some reds, and some random hands would in fact be entirely red, just by chance. If someone claimed to have drawn 10 random hands and they'd all been mainly black, that would be implausible - "too good".Francis's approach is a bit like Uri Simonsohn's method for detecting fraudulent data - they both work on the principle that "If it's too good to be true, it's probably false" - but they differ in their specifics, and I believe that we should not conflate fraud with publication bias... so let's not get carried away with the parallels.Earlier this year, Francis wrote a critical letter about a paper published in PNAS purporting to show that wealthier Americans are less ethical. He argued that the paper's results were "unbelievable" - it reported on the results of seven separate experiments, all of which showed a small, but significant, effect in favour of the hypothesis.Even if rich people really were meaner, Francis said, the chance of 7/7 experiments being positive is very low: just by chance, you'd expect some of them to show no difference (given that the size of the difference in those seven was low, with a lot of overlap between the groups). Francis suggested that the authors may have run more than seven experiments, and only published the positive ones; the authors denied this in their Letter.Anyway, in the new paper, Francis expands on this approach in much more detail, drawing from this 2007 paper, and suggests a Bayesian approach that might help mitigate the problem.Francis G (2012). Publication bias and the failure of replication in experimental psychology. Psychonomic Bulletin and Review PMID: 23055145... Read more »
Francis G. (2012) Publication bias and the failure of replication in experimental psychology. Psychonomic bulletin . PMID: 23055145
Now that the need for more replication has forced its way onto the scientific agenda we should begin thinking about how to build systems to support its growth and institutionalization. New publications and conferences are all good steps, but we should go beyond relying on a loosely organized group of scientists who dedicate time to [...]... Read more »
Francis, G. (2012) Publication bias and the failure of replication in experimental psychology. Psychonomic Bulletin . DOI: 10.3758/s13423-012-0322-y
Today I am going to talk about just one thing rather than poster highlights from the whole day. As always, all the SfN Neuroblogging posts can be found here. Other posts on gender and neurosexism can be found here. Today was the annual "Celebration of Women in Neuroscience Luncheon." This is one of the highlights of SfN for me each year. There is always a fantastic speaker (Phyllis Wise this year) and the lunch is delicious. Phyllis Wise brought up the 'exact same resume study' in her speech and it got me thinking. The 'exact same resume' study is where researchers construct a fake person and write up their resume, and then submit it in application for various jobs. However, sometimes they put a woman's name at the top and sometimes they put a man's name at the top. The study found that the male names received more and higher paying job offers and were judged to be more qualified. And it wasn't just that men thought women less capable. The females who judged the resumes were just as biased as the males who judged resumes. This is pretty depressing. I mean this isn't the middle ages, or even the Victorian era, aren't we past this bias? But that's exactly the problem. We think we are past this bias. Even though people (both women and men) don't think they have a bias, they actually do. Even you. Just like you probably think you are smarter than average, or a better driver than average, you also probably think that you are less biased than average. That. is. the. problem. People have an implicit bias towards thinking men are smarter, better and more capable even when faced with the exact same description of the person. And they don't acknowledge this bias. How can you combat or fix a bias that people don't even think they have? A gender blind resume process could be implemented in the initial application process for a jobs. But as soon as the applicant arrives for an interview, the gender bias would rear its ugly head. Should faculty or hiring committees develop an explicit bias towards women in their hiring and salary negotiation process? I do not know the answer to this question. I can't think of a better way to combat implicit bias than with explicit bias, but it's hard to argue that implementing an explicit bias is 'fair' (It is a bias after all). It would especially seem unfair to those who don't think that they are implicitly biased (which we have established is basically everyone). But is there a fair way to handle this problem?© TheCellularScaleMoss-Racusin CA, Dovidio JF, Brescoll VL, Graham MJ, & Handelsman J (2012). Science faculty's subtle gender biases favor male students. Proceedings of the National Academy of Sciences of the United States of America, 109 (41), 16474-9 PMID: 22988126... Read more »
Moss-Racusin CA, Dovidio JF, Brescoll VL, Graham MJ, & Handelsman J. (2012) Science faculty's subtle gender biases favor male students. Proceedings of the National Academy of Sciences of the United States of America, 109(41), 16474-9. PMID: 22988126
ScienceRewired is a philanthropic initiative that aims to promote public engagement in science through digital and social technologies. Their mission is to aid non-technical science practitioners and the digital domain in working together, to look at science from new perspectives while helping educate and empower individuals to create significant positive change in the world. Their focus spreads across science education, science communication and citizen science initiatives – what’s not to love about that?!(source)I was fortunate to be awarded a scholarship to attend the day in Adelaide (730km / 450 miles away from home) by the Australian Science Communicators. The event was themed ‘Connect, Collaborate and Communicate for Change’ and intended to bring together science communicators, academics, media professionals and digital visionaries for a one day conference of debate, insight and education as a springboard for ongoing communication and action. We heard from a wide range of wonderful speakers about different digital/social media initiatives (most session content has been reported here), but what I wanted to share with you today were two really exciting and different projects that are underway using citizen science.(source)(source)What is Citizen Science anyway?Citizen science has been gaining momentum since the mid-1990’s, but just in case you haven’t heard the term before, relax. You already know what it is even if you haven’t heard the label. Simply put, it’s when amateur scientists or non-professionally-scientific people (i.e. general public) collaborate and help contribute to science. The internet has made this super easy.... Read more »
Khatib F., Cooper S., Tyka M. D., Xu K., Makedon I., Popovic Z., Baker D., & Players F. (2011) From the Cover: Algorithm discovery by protein folding game players. Proceedings of the National Academy of Sciences, 108(47), 18953. DOI: 10.1073/pnas.1115898108
Wiggins Andrea, & Crowston Kevin. (2011) From Conservation to Crowdsourcing: A Typology of Citizen Science. System Sciences, 10. DOI: 10.1109/HICSS.2011.207
Within academic science there has long been a gender bias favouring men over women. Male scientists are more likely to be hired than women despite equivalent qualifications, men get paid [...]... Read more »
Ceci S. J., & Williams W. M. (2011) Understanding current causes of women's underrepresentation in science. Proceedings of the National Academy of Sciences, 108(8), 3157-3162. DOI: 10.1073/pnas.1014871108
Moss-Racusin C. A., Dovidio J. F., Brescoll V. L., Graham M. J., & Handelsman J. (2012) Science faculty's subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109(41), 16474-16479. DOI: 10.1073/pnas.1211286109
In a study that appeared in PLoS ONE two days ago, co-authored by Gustavo Arriaga, Eric Zhou and Erich Jarvis (Duke University), it was shown that a motor cortex region in mice is active during singing, and that it projects directly to brainstem vocal motor neurons that is necessary for keeping song more stereotyped and on pitch.... Read more »
Arriaga, G., Zhou, E. P., & Jarvis, E. D. (2012) Of Mice, Birds, and Men: The Mouse Ultrasonic Song-system Has Some Features SImilar to Humans and Song-Learning Birds. PLoS ONE, 7(10). info:/10.1371/journal.pone.0046610
Why should music be of interest to cognitive scientists, and what role does it play in human cognition? ... Read more »
Pearce, Marcus, & Rohrmeier, Martin. (2012) Music Cognition and the Cognitive Sciences. Topics in Cognitive Science, 4(4), 468-484. info:/10.1111/j.1756-8765.2012.01226.x
When I want to talk to students about how the scientific publishing process can go awry, I almost always end up telling the story of how the Canadian Journal of Physics published a screed against feminism.
In the early 1990s, I was in graduate school when the story broke. The Canadian Journal of Physics had published a set of papers on chaos. Chaos theory was big at the time, following James Gleick’s best-seller Chaos (1987). Among the regular sort of papers one would expect to find in a journal of physics was one very unusual short paper.
First, it was labelled “Sociology.”
Second, it contained exactly two citations. And one reference was to a dictionary for a definition.
Third, the paper was outlandish. It was an almost unimaginable broadside attack on feminism. blaming it for most all of society’s ills. Almost as an aside, it incidentally insulted almost all social science at the same time. To give you a little sampling, this paper claimed:
Women should not be in the work force, because they are“nurturers.”
Half the children of working mothers suffered “serious psychological damage.”
Surveys and controlled experiments in social science distort findings, and “wisdom” is a superior method of obtaining knowledge in the social sciences.
Abstinence until marriage would improve things, which should be vigorously promoted through television advertising campaigns.
It’s rare to find such most blunt “women belong in the home” arguments in any print medium, let alone a scientific journal. It was astonishing.
The article was penned by Gordon Freeman (pictured), who was the guest editor of this one issue of the journal. It was pretty obvious what had happened, in broad strokes: he abused his editorial power to get his poisonous opinion piece into the pages of the journal.
The details of exactly how this happened were a little more complicated. Freeman organized a conference on chaos theory, and was assembling papers that had been presented at a conference for publication in the Canadian Journal of Physics. Apparently, the deal was that the journal would publish all the papers Freeman compiled, provided that they were presented at the conference, and that they were peer-reviewed.
Freeman lied about presenting the paper at the conference.
Somehow, Freeman managed to get back a positive review. The mind boggles, but he did.
The paper was published, and the excrement collided with the rotary cooling device.
Once this story blew up, as was inevitable, the regular editor of Canadian Journal of Physics,
Ralph Nicholls, refused to reveal who the reviewers of Freeman’s
article were, following the standard practice of anonymous peer-review
that most journals follow. Nicholls was removed as editor for refusing
to show the review to the editor in chief for all the National Research
It also came out that Freeman’s institution, the University of Alberta, considered this to be human subjects research, and that Freeman had not obtained approval for it.
The journal apologized for the paper nine months after it was published (Dancik 1991a), but did it in a very uninspired way. The did say Freeman’s work “had no place in a scientific journal,” but it didn’t specifically disavow Freeman’s opinions. And there was no explanation of how Freeman’s paper managed to get into the journal in the first place. Although Science magazine later described the notice as a “retraction,” the word doesn’t appear in the three sentence editorial.
The Canadian Association of Physicists certainly considered it a retraction, and wrote to Freeman asking him to shut up about his paper.
The Editor-in-Chief of the
NRC Research Journals has published an editorial note in the Canadian
Journal of Physics stating that your article has no place in a
scientific journal and expressing regret that it was published. This
clearly retracts any approval of the article by the CJP and, in effect,
retroactively disavows its publication. ... I would ask you, therefore, in view of
the harm engendered to the scientific reputation of the Canadian Journal
of Physics and, by extension, to the whole physics community, to
refrain from making any further references to this article in any public
forum, and not to distribute reprints with the name of the journal
attached. I believe that scientific ethics demand that this article
effectively be struck from the public record(.)
Moreover, the retraction / apology was printed on an unnumbered page, which made it hard to cite, and to link to the original paper. This led the journal to print the apology a second time a few moths later (pictured; Dancik 1991b).
The outrage over this paper was so great that many people were advocating reprinting the entire issue without Freeman’s article, and sending it to every library, so there would be no trace of it left. But a petition to reprint the issue didn’t get any response until the story appeared in Science magazine in 1992 (Crease, 1992).
The Nation Research Council, the publisher of the journal, at one point considered publishing a special issue about the controversy around Freeman’s paper as a mea culpa. But it was not appearing. At one point, the plan not to publish this came out held a conference on the ethics of publishing (reported in Crease 1993; Huston 1993). The uninspired response for why there was no supplement? “It seems a little late now.” This seems to support a claim by newspaper columnist Morris Wolffe on how the controversy was handled:
The NRC, it was clear, was hoping the Freeman matter would go away.
Ultimately, some additional commentary did appear in the March/April 1993 issue of the Canadian Journal of Physics. Proceedings of the ethics symposium appeared in the journal Scholarly Publishing.
If you go to the Canadian Journal of Physics website today to look up the article, this is what you find:
It’s still available, with no indication that this wretched paper has been retracted, or warranted an apology, not once, but twice.
This story was one of the first cases where I became aware of the
concept of retraction. It definitely shaped by view of what retraction
was mean to do. The notion of reprinting the issue set it in my mind
that the goal of retraction was to, as near as possible, expunge an
article from the scientific record. I’ve since realized that in
practice, retraction is a much more complicated beast.
But if ever there was a paper that deserved an unambiguous retraction, this would be it.
Freeman GR. 1990. Kinetics of nonhomogeneous processes in human society: Unethical behaviour and societal chaos. Canadian Journal of Physics 68(9): 798. DOI: 10.1139/p90-116
... Read more »
Freeman Gordon R. (1990) Kinetics of nonhomogeneous processes in human society: Unethical behaviour and societal chaos. Canadian Journal of Physics, 68(9), 798. DOI: 10.1139/p90-116
Environmental problems, such as depleting natural resources, highlight the need to establish a renewable chemical industry. Metabolic engineering enhances the production of chemicals made by microbes in so-called “cell factories”. Next Monday, scientist Professor Sang Yup Lee of KAIST (Korea Advanced Institute of Science and Technology) will explain how metabolic engineering could lead to the [...]... Read more »
Yu Kyung Jung, Tae Yong Kim, Si Jae Park, & Sang Yup Lee. (2010) Metabolic engineering of Escherichia coli for the production of polylactic acid and its copolymers. Biotechnology and Bioengineering, 105(1), 161-171. info:/10.1002/bit.22548
Hey Julie,What a great topic to bring to the table: that what we, as people, like to see in our dogs, may not always be in the dogs' best interests. Indeed. Considering that got me thinking not just about the features we select for when breeding dogs, but also our track record in surgically altering the appearance of dogs through procedures like ear cropping and tail docking, for the purpose of owner satisfaction in how the dog looks.A postcard used in an RSPCA awareness campaign in Australia.Don't worry, I'm not going to show a whole lot of disturbing images, but I would like to talk about the role scientific assessment and objectivity can play as a catalyst for change in regards to animal welfare issues. Pauleen Bennett (source)My PhD supervisor, Pauleen Bennett, founder of the Anthrozoology Research Group and Executive Director and Chair of the Australian Anthrozoology Research Foundation, contributed significantly to ending the stand-off surrounding the issue of 'tail docking' occurring in Australia around ten years ago. She has bred and shown pure breed dogs, so had contact with people and pure-breed dog associations who were strongly opposed to the calls from welfare organisations and veterinary groups to ban the practice of tail docking in Australia. This issue really came to light following anti-docking legislation (except where medically indicated by a vet) being implemented in several Scandinavian and other European countries in the mid-late 1990's. Several main arguments founded in historic practice and emotive reactions were blocking the cessation of tail docking in Australia. Pauleen's review of these issues surrounding tail docking systematically identified and considered the main arguments proposed for continued docking:Maintaining tradition / The breed standard called for a docked tail.The public would not recognise these breeds without their docked tails.The dogs will injure their tails if undocked.The dogs will get dirty tails if undocked.Personal preference (less likely to knock over furniture items, prefer the look, etc.).Within this paper, she also considered the arguments for ceasing tail docking in the same objective and non-emotive manner. The objective and informed facts:Tail docking is painful. It was routinely done to young pups (1-5 days old) who are unable to be anaesthetised (too young) and not given pain relief. Young puppies can feel acute pain.Tails are major communication tools for dogs.Tail docking approx. 500 dogs may avoid one tail injury. (To give this some context with a human flavour, approximately 1 in 255 people fracture their arms annually in the USA, but funnily enough, Americans don’t amputate babies’ arms at birth to avoid this.)Some traditions need to be ceased in light of new information and changing societal expectations/norms. ... Read more »
NOONAN GJ, RAND JS, BLACKSHAW JK, & PRIEST J. (1996) Tail docking in dogs: A sample of attitudes of veterinarians and dog breeders in Queensland. Australian Veterinary Journal, 73(3), 88. DOI: 10.1111/j.1751-0813.1996.tb09982.x
Diesel G., Pfeiffer D., Crispin S., & Brodbelt D. (2010) Risk factors for tail injuries in dogs in Great Britain. Veterinary Record, 166(26), 817. DOI: 10.1136/vr.b4880
Leaver , & Reimchen . (2008) Behavioural responses of Canis familiaris to different tail lengths of a remotely-controlled life-size dog replica. Behaviour, 145(3), 390. DOI: 10.1163/156853908783402894
On the market for scientific jobs, male applicants enjoy a substantial advantage, say Yale University researchers.... Read more »
Moss-Racusin CA, Dovidio JF, Brescoll VL, Graham MJ, & Handelsman J. (2012) Science faculty's subtle gender biases favor male students. Proceedings of the National Academy of Sciences of the United States of America. PMID: 22988126
Many studies on the origins of music concern the question of what defines music. Can birdsong, the song structure of humpback whales, a Thai elephant orchestra, or the interlocking duets of Gibbons be considered music? ... Read more »
Araya-Salas, Marcelo. (2012) Is birdsong music? Evaluating harmonic intervals in songs of a Neotropical songbird. Animal Behaviour, 84(2), 309-313. info:/10.1016/j.anbehav.2012.04.038
People may view this as something for the good news/bad news file, but technology has quietly made it significantly easier to grade tests electronically. For example, a new paper in the Journal of Science Education and Technology highlights a system called “Eyegrade” : While most current solutions are based on expensive scanners, Eyegrade offers a [...]... Read more »
Fisteus, J.A., Pardo, A., & Garcia, N.F. (2012) Grading Multiple Choice Exams with Low-Cost and Portable Computer-Vision Techniques. Journal of Science Education Technology. DOI: 10.1007/s10956-012-9414-8
An illness is an illness wherever you are. Perhaps this is true for organic diseases but the cultural background can play a tremendous role in the progression and even diagnosis of mental disorders. However, what has been neglected is an appreciation of how culture affects the research underlying the diagnosis and treatment of psychological disorders. As a consequence, our view on the disorder can change.... Read more »
Hodgkins P, Arnold LE, Shaw M, Caci H, Kahle J, Woods AG, & Young S. (2011) A systematic review of global publication trends regarding long-term outcomes of ADHD. Frontiers in psychiatry / Frontiers Research Foundation, 84. PMID: 22279437
Polanczyk G, de Lima MS, Horta BL, Biederman J, & Rohde LA. (2007) The worldwide prevalence of ADHD: a systematic review and metaregression analysis. The American journal of psychiatry, 164(6), 942-8. PMID: 17541055
Hey Julie,I’m glad you’re a fan of Paul McGreevy’s too! I admire so much of the work he’s done (and facilitated or helped others to do). He’s not afraid to raise sensitive topics and tricky questions through his science and I appreciate that enormously.Your last post got me thinking about who my canine science (or general doggy-people) heroes are – I’m still pondering on this, but there are plenty of them!(source)I think I’ll get back to you with my conclusions and explanations another day after I’ve had some more time to ruminate on it a bit more.Meanwhile, I wanted to get back to you to talk fat dogs.Sampson, June 2011 (source)One of my local animal shelters was attracting global attention recently when 8 year old Sampson was surrendered to the shelter for rehoming and weighed in at a whopping 84kg (that’s 185lbs). Sampson’s not alone. Up to 40% of Australia’s pet dog population are overweight ad this is comparable to other developed nations. The weird thing is – why? Vanessa RohlfAs pet owners, we humans are in control of their food, we understand that amount of food + amount of exercise = pet weight. We even have ept food companies who have developed weight control formulas so we can feed 'more' but give 'less' - so why can’t some people control the balance to keep our pets healthy?(source)Vanessa Rohlf is a fellow PhD candidate of mine in the Anthrozoology Research Group and she has done some excellent work looking into this area of pet owner/caregiver psychology.... Read more »
Rohlf Vanessa I., Toukhsati Samia, Coleman Grahame J., & Bennett Pauleen C. (2010) Dog Obesity: Can Dog Caregivers' (Owners') Feeding and Exercise Intentions and Behaviors Be Predicted From Attitudes?. Journal of Applied Animal Welfare Science, 13(3), 236. DOI: 10.1080/10888705.2010.483871
Rohlf Vanessa I., Bennett Pauleen C., Toukhsati Samia, & Coleman Grahame. (2010) Why Do Even Committed Dog Owners Fail to Comply with Some Responsible Ownership Practices?. Anthrozoos: A Multidisciplinary Journal of The Interactions of People , 23(2), 155. DOI: 10.2752/175303710X12682332909972
State SAT and ACT scores follow a North-South divide that somewhat fits the spread of racial diversity.... Read more »
Intuitively, the use of police dogs as drug detectors makes sense. Dogs are known to have a better sense of smell than their human handlers. Furthermore, they cooperate easily. Still, compared to the generally good picture sniffer dogs have in the public eye, their performance as drug detectors in real life is terrible. The reason why scent dogs get used anyway holds important lessons for behavioural researchers working with animals or humans.... Read more »
Beran, M.J. (2012) Did you ever hear the one about the horse that could count?. Front. Psychology, 357. DOI: 10.3389/fpsyg.2012.00357
Doyen S, Klein O, Pichon CL, & Cleeremans A. (2012) Behavioral priming: it's all in the mind, but whose mind?. PloS one, 7(1). PMID: 22279526
Hickey S, McIlwraith F, Bruno R, Matthews A, & Alati R. (2012) Drug detection dogs in Australia: More bark than bite?. Drug and alcohol review, 31(6), 778-83. PMID: 22404555
Lit L, Schweitzer JB, & Oberbauer AM. (2011) Handler beliefs affect scent detection dog outcomes. Animal cognition, 14(3), 387-94. PMID: 21225441
NSW Ombudsman. (2006) Review of the Police Powers (Drug Detection Dogs) Act 2001. Sydney: Office of the New SouthWales Ombudsman. info:other/
Hi Julie, Today I want to tell you all about the great time I had last week. But first, you have to know about my lovely friend, Kate. We originally met through our research group at Monash University. We’re colleagues who have completing our PhD’s (both aiming to complete SOON) in common, but we also share a background in Zoology and the fact we have been juggling our research with our work for the past few years. Kindred spirits!This is Kate!Kate works primarily as an animal behaviourist (you can check out her business website, or follow her on facebook), but she is also a media personality, experienced freelance writer and delivers professional and public training workshops. The Melbourne Aquarium asked Kate to come and present to their staff about the ‘Science of Animal Behaviour and Training’ and Kate asked me to come along to help out.Hello penguins!I said yes. Kate’s presentations comprised of theoretical (learning), observational (watching) and practical (doing) exercises. It is always so good to see ongoing professional development in any workplace and the aquarium staff were really engaged and enthusiastic participants. I took lots of fun video footage that you would have seen on our facebook page.Kate’s a great presenter and someone who believes in the fact we (as people) never stop learning. She lives this belief and engages in lots of ongoing professional training opportunities in furthering her understanding of animal behaviour and how we can influence it. For example, she just got back from learning with Ken Ramirez at Shedd Aquarium and over the past few years has also been to Natural Encounters Inc and done chicken camp with Terry Ryan. So what does this have to do with dogs? So much!Animal training specialists understand about animal behaviour (taking into account the individual animal’s history and the effect of the current environment) and the science of learning theory. They use this knowledge to help shape animal behaviour to improve things like animal welfare, enrichment goals and human safety.Divers do group fish feeds in the large aquarium in front of the public. There are big sharks and stingrays in there.(source)... Read more »
Haverbeke A., Laporte B., Depiereux E., Giffroy J.-M., & Diederich C. (2008) Training methods of military dog handlers and their effects on the team's performances. Applied Animal Behaviour Science, 113(1-3), 122. DOI: 10.1016/j.applanim.2007.11.010
In a recent comment in Nature, Daniel Acuna, Stefano Allesina, and Konrad Kording describe a statistical model for predicting h-index. In case you are not familiar with it, h-index is a citation-based measure of scientific impact. An h-index of n means that you have n publications with at least n citations. I only learned about h-index relatively recently and I think it is a quite elegant measure -- simple to compute, not too biased by a single highly-cited paper or by many low-impact (uncited) papers. Acuna, Allesina, and Kording took publicly available data and developed a model for predicting future h-index based on number of articles, current h-index, years since first publication, number of distinct journals published in, and number of articles in the very top journals in the field (Nature, Science, PNAS, and Neuron). Their model accounted for about 66% of the variance in future h-index among neuroscientists, which I think is pretty impressive. Perhaps the coolest thing about this project is the accompanying website that allows users to predict their own h-index.Since hiring and tenure decisions are intended to reflect both past accomplishments and expectations of future success, this prediction model is potentially quite useful. Acuna et al. are appropriately circumspect about relying on a single measure for making such important decisions and they are aware that over-reliance on a single metric to produce "gaming" behavior. So the following is not meant as a criticism of their work, but two examples jumped to my mind: (1) Because number of distinct journals is positively associated with future h-index (presumably it is an indicator of breadth of impact), researchers may choose to send their manuscripts to less appropriate journals in order to increase the number of journals in which their work has appeared. Those journals, in turn, would be less able to provide appropriate peer review and the articles would be less visible to the relevant audience, so their impact would actually be lower. (2) The prestige of those top journals already leads them to be targets for falsified data -- Nature, Science, and PNAS are among the leading publishers of retractions (e.g., Liu, 2006). Formalizing and quantifying that prestige factor can only serve to increase the motivation for unethical scientific behavior.That said, I enjoyed playing around with the simple prediction calculator on their website. I'd be wary if my employer wanted to use this model to evaluate me, but I think it's kind of a fun way to set goals for myself: the website gave me a statistical prediction for how my h-index will increase over the next 10 years, now I'm going to try to beat that prediction. Since h-index is (I think) relatively hard to "game", this seems like a reasonably challenging goal.Acuna, D. E., Allesina, S., & Kording, K. P. (2012). Predicting scientific success. Nature, 489 (7415), 201-202. DOI: 10.1038/489201aLiu, S. V. (2006). Top Journals’ Top Retraction Rates. Scientific Ethics, 1 (2), 91-93.... Read more »
Do you write about peer-reviewed research in your blog? Use ResearchBlogging.org to make it easy for your readers — and others from around the world — to find your serious posts about academic research.
If you don't have a blog, you can still use our site to learn about fascinating developments in cutting-edge research from around the world.
Research Blogging is powered by SMG Technology.
To learn more, visit seedmediagroup.com.