By MERLIN CROSSLEY

From the outside the scientific process appears to entail: deciding on some experiments; doing them; writing up the results; and sending them to the most relevant peer-reviewed journal for publication.

But actually researchers seek to publish their work, not in the most relevant journal, but in the most prestigious journal.

Prestige is a funny thing, and is partly in the eye of the beholder, but broadly it tracks with the Journal Impact Factor (IF). IF reflects how many times papers in the journal have been referenced or cited by other papers in the last two years. No one thinks counting citations is a perfect foundation for evaluating science, nor that papers should be judged by the company they keep in particular journals. But given the competition for resources (grants, fellowships, jobs etc) and the need to make decisions up front long before the importance of each discovery plays out, journal prestige now dominates the thinking of most scientists. In a way it’s horrible – like judging people by whether they went to Eton, Harrow or the local secondary school and by which university they went to rather than by what they learnt.

I do not want to defend the system, but I would say it has arisen in part from a well-intentioned attempt to generate an impartial fact-based meritocracy in science that has ended up having some good but other very unfortunate consequences. By discussing these things and by understanding the nuances, fairer judgements can be made. On top of this, the number of journals has sky-rocketed, so one needs simple guides, including metrics like Impact Factors, to navigate the significance of each.

Many people when they feel they have discovered something that is really exciting consider sending their work to Nature or Science. These are the top two journals in the world. They each publish about fifteen papers a week. Getting a paper into them indicates the editors considered the work to be so important that it would be of interest, not only to people in one’s own field (in my case to molecular biologists) but also to astronomers, geologists, chemists, and even archaeologists etc. The editors are looking for things that would fit on the front page of the New York Times and be remembered as breakthroughs decades later. Things like evidence that Neanderthals bred with humans, or the cloning of Dolly the sheep.

In practice, not everything that is published in Nature and Science changes the world. In some cases, the editors expect it will, but just not yet. In other cases, factors like hype or political networks, come together and some stuff, that glitters but is not gold, ends up making the cut. Nevertheless, even then the work is thoroughly reviewed by top peers in the field and many extra control experiments are suggested. So, in general, the work published in these top journals is reliable

I acknowledge that there are examples where it isn’t. Occasionally people get carried away with excitement. Short cuts are taken in an effort to get into print first, errors are missed, or very rarely, deliberate fraud is used by authors to squeeze their work past the gate keepers. Fraud is the exception and it is not only dishonest but also foolish because the truth usually outs and catches up with people. So, in general Nature and Science are highly regarded and that is why papers in these journals count in the Academic Ranking of World Universities (once known as the Shanghai Jiao Tong League Table). This, together with pre-existing academic competition and even vanity, means that many researchers would like to publish their work in Nature or Science if they can. Even top universities only publish ten or so papers in these journals each year.

If researchers or the editors at Nature don’t think the work is significant enough for their journal they may suggest going one step down, and for molecular biologists like me that might be Nature Genetics. So, what is Nature Genetics?

Late last century as electronic publishing was just beginning Nature did something remarkable. Rather than expanding and diluting their brand by using e-publishing to put out more than ten papers per week, they retained their scarcity restriction and created a new journal called Nature Genetics that would take some of the very good runner-up papers in the rapidly growing discipline of genetics. They later went on to create Nature Structural Biology (now Nature Structural and Molecular Biology), and subsequently other ‘Nature’ titles in almost every discipline (Nature Climate Change, Nature Geoscience, Nature Medicine, and even Nature Reviews etc.). Publishing in these journals is not as prestigious as publishing in Nature but it is still impressive, and the whole Nature stable has managed to retain its prestige.

If the editors of Nature Genetics feel the work is not of broad enough interest for them – perhaps it would not make it onto page two of the New York Times – they might suggest Nature Communications.

Nature Communications was another publishing master stroke for Nature that also took advantage of a new market opportunity.

In the past if your work wasn’t accepted in Nature or Science researchers would often try the respected general journal, Proceedings of the National Academy of Science, USA, or PNAS – which wags dubbed “Probably Not Accepted in Science”.

Nature realised that there was so much good science now that there was room for another general interest journal, and they invented Nature Communications and made it open access, charging people relatively high production costs (or page charges) to support its operations and turn a profit. It worked. Researchers were happy to publish there because again the journal leveraged the Nature name. Meanwhile Science took a similar approach and created Science Advances, and another top journal Cell, created Cell Reports. In my own discipline, the American Society of Hematology has a top journal called Blood, but has now introduced a new journal Blood Advances which takes the high quality overflow papers. Beyond the general journals, there is a rank of specialty journals in every field. Some of these are run by big publishers like Nature (some in partnerships with universities, like our UNSW-Nature partner journal Quantum Information). Other specialist journals are run by academic societies, which make a major high quality contribution, but have been unable to keep up with the volume of modern publishing.

To complete the story I should say that Nature also launched a journal called Scientific Reports, which like another newish journal, Public Library of Science One, or PLoSONE, takes papers that are sound and are professionally done but may not quickly change the world. I like the fact there are journals available that do not judge the science by how broadly interesting the findings are, as that they ensure all good experiments can be published somewhere and reduces the pressures people feel to sell, or oversell, their work. Some people like to list Scientific Reports on their CVs as Nature Scientific Reports, to leverage the Nature name, but doing that is generally frowned upon.

Likewise, saying we recently had a paper in Nature, when it was in fact in Nature Genetics, is also considered poor form but people sometimes get away with it because those outside the research community may not appreciate the difference. Interestingly, Nature Genetics has sometimes had a higher Impact Factor than Nature but this is because the genetics community is very large and generates a lot of citations, whereas the community that appears in Nature involves not only geneticists but also geologists, chemists, physicists, etc and overall there can be slightly fewer citations. Nevertheless, Nature, Nature Genetics, and Nature Communications are all very highly regarded. Scientific Reports is fine but a paper there is much less likely, in itself, to make your career.

Some people lament the fact that for the sake of enabling decisions related to resource allocation and career advancement serious scientists are forced to navigate this cascade of journals, and that general interest (rather than just the soundness of the science) has such a strong role to play. Understandably people point out that scientists can not control whether their hypothesis is right or not, and doing solid, rather than flashy, science should be the goal, or put another way, journal rankings and Impact Factors shouldn’t matter.

I guess this is another triumph of popularism, a bit like the drift from test cricket to the big bash. Publishing in Nature is like hitting a six (over the fence), Nature Genetics is a four (a boundary), Nature Communications is a hard run three, and Scientific Reports is the equivalent of diligently blocking and taking the occasional quick single, without losing your wicket.

In any one innings in a test match you wouldn’t expect a batter to score more than one or two sixes, and indeed few researchers publish more than ten papers in Nature during their careers. The crowd likes to see sixes though, so it’s hard to get into the team or keep your spot and maintain your research funding if you never score a six or a few boundaries, even if you are not bowled out. These days the competition for research funding is intense and panels will tend to look for at least a few big hits in recent times. Consequently, this publishing cascade has become hugely important and savvy researchers carefully balance their activities to ensure they hit a six sometimes and at other times take an easier quick single.

One of my colleagues insists that judging people by where they publish is as illogical as judging cricketers by whether they play at the Melbourne Cricket Ground (MCG) or not, rather than by considering how many runs they score. But I disagree and point out that you have to do both and you can judge cricketers by how often they have played at the MCG and how much they scored when appearing on the top grounds. Like it or not, researchers know that it matters where you publish.

But if we are to value publications in top journals, it’s important to keep some things in mind. The reality is that, although the editors try their best to be fair, if you are in a big lab, with a long record of contributions in the Western hemisphere your work probably is more likely to be taken seriously by the editors and sent out for peer review. And when the reviewers’ comments come in people in big well-funded labs will be better placed to tackle the many, many additional control experiments that are required.

If you are in a smaller lab in a non-English speaking country that has not yet produced important work then there is a greater chance you will not be able to convince the editors your work is as good as you say. And even if you do, limitations with respect to resources may make it harder for you to complete all the new control experiments required by the reviewers.

In other words, when job committees or selection committees consider track records they should be mindful of the environments and assess achievements relative to opportunity. On the other hand, even if you are in a Nobel Laureate’s lab it still will not be easy to get a paper into Nature, so I do not want to take anything away from those that succeed, just to acknowledge that context is important and we should always be mindful of that. I would also say that although the systems are imperfect there probably is value in people aiming for excellence in research and planning to hit a six out of the park every now and then.

Merlin Crossley

Deputy Vice-Chancellor Academic, UNSW


Subscribe

to get daily updates on what's happening in the world of Australian Higher Education