Archives

Categories

Add-ons to Science Survival Guide book


Klaas Wynne Klaas Wynne 18 February 2009

Where to publish (a 4* post…)

Tags: , ,
Posted in Getting published, High-impact journals

Last year, the UK had a giant review of all its university departments to arrive at rankings of departments by subject. This review was called the research assessment exercise (RAE) 2008 and my department (a physics department) didn’t do so well. Therefore, I had an extra good look at the RAE results. In January, we got some more details including a ranking of our papers. Each academic had submitted four papers published between 2001 and 2008, which were graded by a panel from 1* to 4*. The meaning of this ranking is 4* (world-leading), 3* (internationally excellent), 2* (internationally recognised), and 1* (nationally recognised). From my department’s result, I could work out a formula relating the impact factor (IF) of the journals to the quality of the paper as judged in the RAE 2008. The Physics panel chair Sir John Pendry vehemently denied a few weeks ago that his panel used IFs. That may be true but then my formula calculates the perceived quality of a paper as judged by our peers. I thought you might be interested in that judgement.

The formula is:

4*: IF > 10
3*: 6.5 < IF < 10
2*: 2.5 < IF < 6.5
1*: IF < 2.5

Thus, to be seen as ‘world-leading’, you need to publish in Science or one of the Nature journals. The ‘internationally excellent’ journals range from Phys. Rev. Lett. in physics, J. Am. Chem. Soc. in chemistry, to Adv. Mater. in materials science.  The ‘internationally recognised’ journals are the bread-and-butter of science and include Phys. Rev. A, B (but not E), J. Chem. Phys., Appl. Phys. Lett., Opt. Express, etc. Journals with an impact factor below 2.5 are not worth publishing in within the RAE scheme.

One thing to add: of course, one should take into account citations too. Thus, a paper published in Acta Crapta, which gets 10-15 citations within two years of publication, should be regarded as 4*. Similarly, a paper in Science, which garners no citations after two years or more is 1*.

Related to all this is an editorial published on 9 February 2009 by two Associate Editors of Phys. Rev. Lett. where they explain that if they only published the top 500 letters, their impact factor would be 20 instead of 6.9.

So, what is the conclusion after all of this? Well, first you have do good interesting science and write it up well, that is, in an interesting fashion, so that it is clear to all that your paper is exciting. Then, you send it to the appropriate journal. Probably not Acta Crapta or the Journal of Irreproducible Results…

Email, RSS Follow

Related posts:

  1. Where to publish an article?
  2. Our readers want to know everything about impact factors
  3. Should scientists make their own drawings and illustrations?
  4. Critical article on the H-Index
  5. Should editors of Science and Nature socialize with scientists?
  6. “The nature of the contribution of every author should be made clear”
- - - - - -
If you like this post why don't you email subscribe to our new posts. Or subscribe to our RSS feed.
  1. Unregistered

    26 Feb 2009 14:23, Jacopo Bertolotti

    Like many indicators the Impact Factor has plenty of flaws and is prone to fail badly on the single case. Nevertheless it has some validity and it is somehow “self-fulfilling”. I’ll try to make myself clear with an example: if I have a nice result I try to go on the best possible journal (defined as the journal that will look better on my cv) but at the same time I try not to shoot too high ’cause I’ll be rejected and I’ll just loose time. Therefore if I have a modest result I’ll go for low-profile journal, if I have a nice result I’ll go for Phys. Rev. Something, if I have a very good result I’ll go for Physical Review Letters and (finally) if I have an extremely good result I’ll go for Nature/Science.
    Of course I might manage to get my “not so great” paper accepted in Nature or I might publish a world-shacking result in Josa A, but on average the correlation “work level” to “IF of the journal” is quite high.

  2. Unregistered

    1 Mar 2009 9:34, Bram van Ginneken

    When you quote from the PRL Editorial, you leave out an essential caveat. Here’s what the editorial says: “By contrast, the 500 most-cited Letters—if published separately, and again assuming one could spot them a priori—would have an impact factor of around 20.”

    I think spotting highly cited papers a priori is much more difficult than many critics of impact factors, h-factors, etc may think. They seem to think that if they’d just focus on the trendy areas, citations will just flow in. They forget that in those areas, a lot of papers are published and only some of these are highly cited. I have not been very successful, in retrospect, in predicting which of my papers would attract the most citations.

  3. Unregistered

    1 Mar 2009 20:16, Klaas Wijnne

    @Jacopo: Yeah, sure, you are quite right. In fact, what happened is that I made up a formula that I thought made sense to me and then tweaked it ever so slightly to make it match the RAE 2008 results. The only real tweak was to move the 1*/2* boundary from an IF of 2 to 2.5. Although I agree with you, it is interesting to note that the RAE panel only looked at 50% of all papers, which means that the other 50% must have been assigned a score by automatic means…

    @Bram: I agree with you there too. There might be bit of a self-fulfilling prophecy going on there though: people assuming that if something has been published in Nature/Science it must be really good and gets cited loads.

    Another thing: an interesting thing was that my colleagues in my department were quite upset with my formula. Like, how could I possibly suggest that they might have to publish in PRL or even, gulp, Nature?! Most seem to think that Opt. Commun. is still one of the top journals in the field of optics.

  4. Unregistered

    21 Mar 2009 0:15, Otto Muskens

    One aspect I am missing in the IF is normalization to the size of the community. A journal such as Nano Letters which caters to both chemists and physicists may attract a lot more citations than a core physics journal like PRL. This is not corrected in counting the IF. So maybe it is good if one is capable of raising interest of a large interdisciplinary community. But does the size of the community determine the value of one’s work? This would mean we should stop doing research in smaller core fields and all follow the mainstream and hype research. Maybe yes, but I am not so sure. There should be an optimum in there.

    By the way, our university also scored bad in the RAE. It should be noted though that the difference between positions 7 and 15 on the ranking was only a difference of two stars in a total of 180 papers…

  5. Mirjam

    21 Mar 2009 15:53, Mirjam

    It is true that the impact factor will depend on the size of the community that a journal caters to, but it would be hard to correct for this as it would require an exact determination of where one field ends and another starts. It would also be a bad idea for scientists to use the potential impact factor as the sole motivation to start a particular line of research (unfortunately, they sometimes do). As the use of some kind of scale to measure someone’s productivity is inevitable in the present day funding structure, I think the only right conclusion is that the impact factor is not an absolute indicator, but can be a good guide if it is used wisely and seen in the context of the pertaining field. It is the task of the funding organizations and review panels to ensure this.
    Even so, there will still be scientists that within their field act as brainless impact factor hunters, i.e. people that send their every fart to Nature or Science and sometimes are lucky enough to get a mediocre paper published in these journals. Personally, I find this seriously bad practice, because it wastes resources and time of the editors and reviewers, devaluates other good papers in the same journal, and leaves a stain on the image of the field. I know it would be almost impossible to implement, but it would be great if there were a mechanism that deters this ‘shotgun’ approach. Something along the lines of multiple choice exams in which gambling is discouraged by assigning a good answer +2 points, a wrong answer -1 point and not answering 0 points.

  6. Unregistered

    22 Mar 2009 16:40, Klaas Wynne

    Otto, Mirjam: I agree that the size of the community will part determine the IF. Another factor is the typical size of the author lists. In fields such as particle physics, papers often have as many as 100 authors. If all (or a majority) of these authors go off to write other papers they will likely cite their own work, nearly automatically resulting in 100 citations. So clearly, funding agencies etc. should correct for such things. I’m not sure if I quite agree with Mirjam’s “brainless impact factor hunters” comment. Do you actually know of any examples where somebody has sent a large number of manuscripts to Science/Nature and got a poor one accepted? Anyway, personally I think it is healthy to think about what makes the broadest impact, what is important to science in general and your chosen area, and to adjust your own work accordingly. That does not make you an IF slave, just a more intelligent researcher. As for the RAE, Otto, your department didn’t do bad at all: 15th shared with places such as Oxford. We came 34th…

  7. Otto Muskens

    22 Mar 2009 23:00, Otto Muskens

    Klaas, I agree with you that it is healthy to think about impact of your work in a broad context. However, with the advent of purely bibliometric-data based evaluation systems such as the REF, playing the game will become more important than doing good science.

    It is only a matter of time before somebody will publish the ‘Science survival metrics tweaking guide’… This person will probably make more money than Ad did.
    About the RAE: mind that Southampton ranked 5th in 2001, which makes it a dramatic performance according to the local authorities (with financial consequences).

  8. Mirjam

    23 Mar 2009 13:02, Mirjam

    I completely agree with Klaas’ statement that ‘… it is healthy to think about what makes the broadest impact, what is important to science in general and your chosen area, and to adjust your own work accordingly.’ Unfortunately, I know a couple of scientists that forget about the ‘what is important to science’ part and submit everything to Nature/Science first in their hunt for high IF. What makes it even worse is that afterwards they think it is quite amusing that they got something published in these journals that actually doesn’t really belong there and will joke about it a lot… (I know of a recent example, but don’t want to point to specific people). In my opinion this ruins the game for serious researchers that want to publish in these journals.

  9. Unregistered

    17 May 2009 22:03, Successful Researcher: How to Become One

    Thanks! It’s interesting to learn how things are in the UK.