Buy

Books
Click images for more details

Twitter
Support

 

Recent comments
Recent posts
Links

A few sites I've stumbled across recently....

Powered by Squarespace
« Global sea ice normal | Main | Chris Rapley goes all fair and balanced »
Wednesday
Mar242010

More peer review gatekeeping

Icecap has an interesting new article by three sceptic scientists - John McLean, Chris de Freitas and Bob Carter - describing the successful attempts to deny them a right of reply in the peer-reviewed literature.

The practice of editorial rejection of the authors’ response to criticism is unprecedented in our experience. It is surprising because it amounts to the editorial usurping of the right of authors to defend their paper and deprives readers from hearing all sides of a scientific discussion before they make up their own minds on an issue. It is declaring that the journal editor - or the reviewers to whom he defers - will decide if authors can defend papers that have already been positively reviewed and been published by that same journal. Such an attitude is the antithesis of productive scientific discussion.

Read the whole thing (PDF).

PrintView Printer Friendly Version

Reader Comments (19)

Dear God. I have never heard of a Response being refused. I can see that an Editor might want to vet it for, say, length He might even want to insist on civility. And he's duty-bound to require clarity. But since it passes all three of these tests, I think that the guys have a legitmate complaint. Shameful.

Mar 24, 2010 at 12:08 PM | Unregistered Commenterdearieme

So, let us create a list of publishers with unscientific behaviour or doubtful scientific behaviour. Or a table with publishers and grading their scientific behaviour and thinking (not to mix up with the quality of the science they publish): 1-6:
Scientific above all means
Very small scientific flaws
Not always scientifically correct
......

Then a second list to be created by exceptional staticians:
statistically beyond doubt
minor flaws
statistics dead wrong

And another
data cherry picked - or not

And a list of publishers who or whose peers accept sentences like: We believe in ......., especially in the abstracts. Belief is not science.

Then a list of peers judging in circles: eg. Jones-Mann-Trenberth-Jones-....


Be the first one to rate publishers and they will get into thre defense position. My 2 cents. Keep on, you'r doing fine.

Mar 24, 2010 at 12:29 PM | Unregistered Commenteregp

To add: The goal of such lists is not to get all publishers or partisan publishers in a defense position, but to mark those who act unscientifically, like those who refuse responses as is described above.

Mar 24, 2010 at 12:33 PM | Unregistered Commenteregp

Sigh! How utterly sad.

Mar 24, 2010 at 2:13 PM | Unregistered CommenterDon Pablo de la Sierra

This is really utterly depressing. The state of climate "science" is dire....

Besides who's scientifically right or wrong, the process is rotten until the very core.

Mar 24, 2010 at 3:33 PM | Unregistered CommenterHoi Polloi

No surprise to see Tamino misbehaving here.

Mar 24, 2010 at 4:17 PM | Unregistered CommenterDavid S

This is disturbing. It makes the issue of the make up of various investigatory bodies even more of an issue. Jones' involvement in this incident certainly needs to be brought to the attention of Russell et al.

Mar 24, 2010 at 5:22 PM | Unregistered CommenterBernie

Appalling.

Mar 24, 2010 at 6:13 PM | Unregistered Commentermondo

There is no "right of reply". This is a scholarly journal not a university debating club.

The reason they couldn't get a reply past peer review is the obvious: their attempts at replies were crap, as was their paper.

James Annan has responded to their 'science by press release'.

Mar 24, 2010 at 8:57 PM | Unregistered CommenterFrank O'Dwyer

James Annan has a different take to be sure, but methinks he doth protest too much.

Mar 24, 2010 at 11:34 PM | Unregistered CommenterBernie

This bears certain similarities to the issue of the Mckitrick and Michaels examination of the surface record. Gavin Schmidt published a critique of it in the IJOC which contained two erros. The first was that he confused autocorrelation in the dependent variable of a regression, not necessarily a problem, with autocorrelation in the residuals, which does matter. And M&M's equation did not have autocorrelated residuals. The second errors was that Schmidt did not realise that his own estimated equation, alleged to refute M&M, actually had coefficients which lay outside the range which he himself said would constitute refutation. Mckitrick drafted a response pointing out these mistakes, in my opinion devastating and unanswerable, which IJOC has recently rejected on the flimsiest of grounds. YOu can find all the relevant material here

http://sites.google.com/site/rossmckitrick/

in the new items section.

Note that Phil Jones gave a very cursory review of the Schnmidt paper - see climategate emails, which failed to pick up either of the mistakes and also seemed driven by the idea of getting a refutation of M&M into the peer-reviewed literature before the AR5.

Mar 24, 2010 at 11:42 PM | Unregistered Commentermikep

Check Appendix A Page 12 of the Critique
Last paragraph
(b) inappropiate splicing of different data products

Mar 25, 2010 at 1:12 AM | Unregistered CommenterLarry

Frank O'Dwyer - you said

QUOTE
There is no "right of reply". This is a scholarly journal not a university debating club.
The reason they couldn't get a reply past peer review is the obvious: their attempts at replies were crap, as was their paper.
UNQUOTE

The appropriate method would have been as follows:
Scientist A has a paper published after favourable peer review.
Scientist B disagrees with the conclusions and has a paper published after peer review, which criticises the first paper.
Scientist A replies, rebutting these criticisms to the best of his ability. This is also peer reviewed.
THEN the scientific community evaluates these papers and individual scientists come to their own conclusions.

Refusing to publish the rebuttal distorts the scientific process and reflects badly on the integrity of the journal.

Mar 25, 2010 at 6:23 AM | Unregistered CommenterAusieDan

Frank O'Dwyer:
There is no "right of reply".

Formally, there might be no right to reply but there was an invitation to a reply by an editor.

November 20, 2009. Editor-2 informs McLean et al. that:
“Comments on your JGR-Atmospheres manuscript "Influence of the Southern
Oscillation on tropospheric temperature" have been submitted by Foster et al.
and have passed peer review. I invite you to submit a reply to the comments,
which are attached. ...... Your reply will be reviewed and, if acceptable, will be
published at the same time as the comments.”

I have read the comment by Foster et al. and the unpublished reply to the comment.
The comment appears to me well in line with all the (sometimes weak) rebuttals by the Hockey Team (B.H.'s HSI book contains some other pretty examples).
The usual tactics is: Claim that an author has said something, which he actually has not, and rebut that. Then this author needs to spend a lot of time (and journal space) to make clear what he has originally stated and to express, politely, that the commenters have obviously not read the paper clearly. And the original paper is, in this case, not to blame as it is indeed quite clearly written.

Frustrating, if people do not read your papers as carefully as you have written them before declaring them wrong. Politicians behave like that, some scientiest obviously as well. Listening is a virtue of debating, well, unless you don't have a point but don't like what your opponent is saying.

Sad...

Mar 25, 2010 at 9:20 AM | Unregistered CommenterStefan

Frank O'Dwyer

It is beyond belief that you can even consider defending behaviour such as that outlined in the SPPI paper. You may, in your wisdom, think the paper is crap (doubtless based on the opinion of team members), but at least 3 qualified reviewers disagreed and recommended publishing.

That is what peer reviewed means. If this was a paper supporting your prejudices, you would declare that it was Peer Reviewed, and therefore true beyond all question.

JGR has clearly breached both their own publishing guild-lines, and what any fair minded observer would regard as proper scientific conduct. Your defence of such behaviour condemns you as a scientific corruption denialist, who will condone any degree of scientific malpractice as long as it supports your pre conceived opinion.

Mar 25, 2010 at 11:20 AM | Unregistered CommenterPeter Wilson

AusieDan,

Scientist A replies, rebutting these criticisms to the best of his ability. This is also peer reviewed.

And that is exactly what occurred. The best of their abilities wasn't good enough. It didn't pass peer review.

What on earth do you think the function of peer review is, if it is never to reject anything?

They are not being censored. Their reply is published, just not in the peer reviewed literature (where only things that pass peer review get published, the clue is in the name).

Indeed there is probably not a scientist in the field, and many a non-scientist, who isn't aware of their reply. Just now they are aware of one other salient fact: they could not, or would not, muster a reply that would pass peer review.

Mar 25, 2010 at 8:37 PM | Unregistered CommenterFrank O'Dwyer

Frank O'Dwyer

It is hard to believe you are seriously suggesting that the reply in question was so deficient as to be unworthy of publication, and that this unworthiness outweighed the obvious bias betrayed by succumbing to pressure from Jones et al to leave a heavily biased criticism (which struggled to pass peer review itself), unanswered.

Of course the problem with this idea is that we have all read the paper and the reply, and this is clearly not the case. You mention no specific reason why the reply was so unworthy, other than Phil Jones' say so. At the very least the scientific community should have been afforded the opportunity to assess the validity or otherwise of the arguments. To not do so strongly implies a desire to avoid scrutiny, which is often the mark of shoddy science.

Mar 26, 2010 at 4:21 AM | Unregistered CommenterPeter Wilson

Peter Wilson,

You mention no specific reason why the reply was so unworthy, other than Phil Jones' say so.

I have not mentioned Phil Jones at all, you appear to be hallucinating. The reply was clearly unresponsive to the criticisms in the comment. I also note that McClean et all quote approvingly from one of the comment's reviewers:

But as it is written, the current paper almost stoops to the level of "blog diatribe". The current paper does not read like a peer-reviewed journal article. The tone is sometimes dramatic and sometimes accusatory. It is inconsistent with the language one normally encounters in the objectively-based, peer-reviewed literature.

While neglecting to quote the rest of the reviewers comments, i.e.:

Accept pending major changes (mainly in style not scientific comment)

The real mystery here, of course, is how the McLean et al. paper ever made it into JGR. How that happened, I have no idea. I can't see it ever getting published through J Climate. The analyses in McLean et al. are among the worst I have seen in the climate literature. The paper is also a poorly guised attack on the integrity of the climate community, and I guess that is why Foster et al. have taken the energy to contradict its findings.

So the current paper (Foster et al.) should certainly be accepted. Someone needs to address the science in the McLean et al paper in the peer-reviewed literature. But the current paper could be - and should be - done better. That's why I am suggesting major changes before the paper is accepted. All of my suggestions have to do more with the tone and framing of the current paper, rather than its content.

Curious, eh?

Mar 28, 2010 at 8:13 AM | Unregistered CommenterFrank O'Dwyer

Frank O'Dwyer

The problem with your defence of all this is that the climategate emails clearly show the intrigue behind stacking the review panel with Team members. The comment you quote reads eerily like some of the comments in the emails, and it would be unsurprising if they had the same author.

The reply was not at all unresponsive to the criticisms, pointing out that they were misdirected because of a basic misunderstanding of the purpose of the statistical techniques used . Once again you comments indicate that you believe you are addressing people who have not had the opportunity to examine the documents in question. Your comment is simply untrue.

Mar 28, 2010 at 3:11 PM | Unregistered CommenterPeter Wilson

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>