Sanli
13 May 2008
Proposal is unacceptable. He wears red shoes.
Posted in EthicsA young scientist in his early thirties, with more than 40 publications and a permanent position at a European university, has gotten the following comment in the judgment of his research proposal: the candidate writes articles with many different coauthors. And, that is listed in the weak points of the proposal.
I assume a scientific proposal is to be judged based on its scientific value, and not based on the personality of its author. The number of coauthors relates very much to the history of a scientist’s activities and has nothing to do with his scientific qualifications.
Either weak point or strong point, the above mentioned comment of that referee is certainly a textbook example for a formal fallacy of the ad hominem type, which consists of criticizing or personally attacking an argument’s proponent in an attempt to discredit that argument.
How can an established professor, a messenger of scientific argumentation, state such a fallacious reasoning? Is it the power of being part of the elite, or is it a side effect of anonymity?
It takes you just few minutes to learn about several types of fallacies in wikipedia.
Follow
16 May 2008 11:44, ad lagendijk
Can one appreciate (judge) a painting without knowing the painter? In art and literature the posing of this type of question invariably causes a heated debate. In science the idea of double blind refereeing (anonymous referees and anonymizing the authors) of submitted papers is regularly put forward as the best form of peer review. Advocates claim that the unjust advantage of papers, coming from established institutions, or written by established scientists, could be neutralized in this way. Given the fact that almost no scientist can resist the temptation of multiple self-citations it would be quite difficult to maintain the anonymity of the authors in practice.
So why is a proposal not judged by only looking at the quality of the proposal? Why are other factors, including social aspects, related to the applicant, included when reviewers evaluate the proposal?
I think the answer is quite simple. Proposals are in a way promises. Promises made by the applicant to the grant organization that he will carry out a certain type of research if he would get the requested support. But how is this funding organization to know that the principal investigator will keep his promises. He might have a record of a notorious promise breaker or she might be known as a meticulous promise fulfiller. For this type reasons the past performance of an applicant is always taken into account by grant proposal reviewers.
How about the particular case of too many coauthors? This putting down of social skills depends very much on the field. In some fields a paper typically lists hundreds of authors. But in other (Little Science) fields a paper has in the majority of case less than six authors.
Little Science
In the little-science disciplines reviewers like to see applicants to have a number of first-author-papers and possibly a number of last-author papers. If scientists have many papers on their cv and they are always in the middle of the author list reviewers might conclude the scientist is not very original. If the grant proposal is about embarking on new, original research I certainly understand why this factor is hold against the applicant. A typical example is a beam-line scientist. That is a person who is working at an international facility and functions as the scientist who is supporting the experiments of users of the facility. The beam-line scientist will always be on the papers (somewhere in the middle of the list), although he hardly has contributed any intellectual content.
16 May 2008 17:39, Sanli
Thank you Ad for describing the reason behind judging the performance of the applicants. I still believe that funding organizations should not accept qualitative and vague arguments from the referees. Based on qualitative arguments, it is hard to check the consistency of one referee with himself in judging different proposals, or two referees with each other. Without consistent criteria, selection of proposals based on referees’ incomparable comments does not really differ from flipping a coin; it suffers from an equal amount of error.
As an example, let’s take the above discussed argument. It could have been more reliable if the referee had made it quantitative: “The candidate has been a middle author in n% of his publications. I describe a scientist that has been a middle author more than 50%, as a beam-scientist, therefore…” If the referee reports such numbers, he will not be allowed to accept another proposal in which the candidate is qualified the same. If all the referee’s report their opinion in the same way, it is possible to check for those who give an out-of-norm weight to this specific factor in their judgments.
p.s. Not related to our discussion, but the scientist of my story is 90% the first author in her articles. His papers have in average 2 authors.
7 Aug 2008 21:17, outsidethebox
It should be judged on it’s own merit of content. Perhaps there is more to it, the referee may have other reasons.