What the Hell?
Can anyone give me the ten-second elevator speech about this latest climate controversy–the one involving Eric Steig and Ryan O’Donnell? That would be the speech going up.
Then on the way down, can anyone give me another primer on why this dust-up matters?
I’m serious. When even this guy calls it an “incredibly complicated story” then I know it must be complicated.
For bonus points, please direct me to any sites that you think present this “incredibly complicated story” in a fair and evenhanded manner.
UPDATE: Steig responds in full to the controversy over at RC. Will it quiet the mob? I’m sure we’ll find out in no time.
UPDATE: Andrew Revkin has a nice meta overview post at Dot Earth.
I’d suggest everything at Lucia’s for starters.
Guy A says Guy B gamed peer review to make Guy A’s paper look dumb. Guy B says “nah, I didn’t do it, man, you just wrote a dumb paper.” Not complicated. One guy’s word against another’s. Anybody who says they know what really happened is making assumptions based on allegiances preconceived notions.
Not complicated at all.
Gryposaurus lays out the timeline at Bart’s: see his comment at February 9, 2011 at 21:10 et seq.
In the newest RealClimate post Steig says O’Donnell has offered to retract his claims
http://www.realclimate.org/index.php/archives/2011/02/odonnellgate/
That O’Donnell’s paper was praised by Steig for getting the same results and and offering improvements on his own paper whilst being called dumb shows the flexibility of interpretations here. Confirming that Steig was Reviewer A was not too much of a surprise, but the amount of leeway Steig was given by the review editor to force changes out of O’Donnell’s paper is a bone of contention. Especially when you contrast his response to the other two more favourable reviewers.
Shorter Ryan O’Donnell.
Layman summary:
Steig, as Reviewer A suggests that iRidge is better than TTLS, which then results in a rewrite in which iRidge is used in the published paper and TTLS is moved to the supplemental material. Both show Steig’s method wrong.
After paper published, Steig criticizes the use of iRidge, which he suggested was the best method as Reviewer A.
O’Donnell gets irritated at hypocrisy and goes public with Steig’s identify as Reviewer A and publishes all the review comments and responses.
Steig then publishes a post at RC saying that he never saw the last response O’Donnell made to the journal – so he is not really a hypocrite.
Well, now that this is all sorted out, it’s time we moved no to the next mudfight at the Okclimate corral, no?
Seriously, this latest episode was so ugly and one-sided (in terms of not hearing a response from the other side) that something told me to wait a bit, and even then, it wasn’t clear to me what all the ruckus was about.
But for those not persuaded by Steig’s explanation at RC, go ahead and make your best counter.
Keith Kloor Says:
February 9th, 2011 at 5:25 pm
“But for those not persuaded by Steig’s explanation”
There is the matter of the 88 page critique Reviewer A made of the original O’Donnel paper. Of course O’Donnel was refuting part of Reviewer A’s 2009 paper.
The referee’s already had to be called in on the paper prior to publication. Looks like the brawl spilled out onto the street.
Keith, I’d still play the waiting game on this one. I think more is going to come out and I think it’s distinctly possible both sides will look a bit ugly while feeling innocent and insulted. It shapes up to be a real mudfest.
That’s okay, right? Peer review hasn’t had any criticism recently…
Tom, I’m not suggesting Steig’s is the final word. But I do feel many people jumped the gun on this, based on incomplete information.
I personally did not feel comfortable weighing in on this–and still don’t.
Steig puts O’Donnell through hell from a position of power an an anonymous reviewer. O’Donnell works very hard to comply with Steig’s requests/suggestions/demands. O’Donnell finds out Steig is the reviewer but promises silence. Steig then publicly criticizes the paper for making the very changes that he requested/suggested/demanded. O’Donnell who feels justifiably aggrieved, loses his cool, betrays his word and outs Steig as the reviewer. Steig points out that while he “agreed” to the changes in the paper, he still had the right to criticize it. Personally, I think Steig should never have been the peer reviewer on a paper that a response to his own. If he was going to publicly criticize methodologies in the paper, he had a moral obligation to explain his editorial role. O’Donnell on the other hand had an obligation to keep his word and to keep the conversation professional and civil. Frankly, both sides lost.
Much depends on whether O’Donnell can convince the blogosphere that the original Steig, S09, was badly flawed. If he can, and if Steig put O’Donnell ‘through hell’ for correctly criticizing his paper… The rest won’t matter much.
Steig shoulda maybe recused himself. But, hindsight and all that.
What drama.
Keith, this is about abuse of the peer review process.
Climategate was about abuse of the peer review process.
Stieg et al proves that nothing has changed, AGW has been sustained by abuse of the peer review process.
Your call, it is your site after all!
ps I have been waiting for your call on this story, along with Tom Fuller’s, as I respect your views. Tom? Keith?
Imagine a chef at a fine steakhouse. A customer comes in and order the filet mignon extra well done. He informs you that he is on a low salt diet so you don’t put salt on his steak. He substitutes dry macaroni for the double stuffed potato that normally comes with the meal. You comply with his requests. Later you find he owns your biggest competitor and he is reviewing your restaurant in the paper. The steak was overcooked and dry. It lacked salt. Dry macaroni was a bland side dish for the steak.
The chef is livid. The customer replies “but is anything I said untrue?” The chef punches him out. The customer shouldn’t be reviewing with a conflict of interest and certainly not criticizing without revealing his role in the creation of the dish. However, this doesn’t give the chef the right to punch him.
Keith, it’s the editor’s call whether Steig should have been a reviewer or not; it’s the editors (or a sub editor) who assigns reviewers. O’D et al likely also got a chance to put Steig on a ‘please do not send to these reviewers’ list, when they first submitted. Whether to accede to that request is also the editor’s call. It’s certainly not unprecedented for the author being critiqued, to be a reviewer (one can then expect a mighty detailed review…which I think we can agree, O’Donnell et al got). That’s the way it works in science, at least in my field.
As for the drama, I kinda doubt Antarctica cares. It’ll be interesting to see what happens to the western peninsula in the next decade.
Oh, and JonChe #15, your analogy, in addition to being far too contrived and convoluted to be effective, is also not apt — the ‘customer’ (Steig) didn’t ‘order’ the substitutions (iridge) to the ‘steak dinner ‘ (O’donnell et al’s methods). It was the ‘chef’ (authors O’D et al) who suggested them. Steig merely agreed that it would be a good idea, so long as the authors address the known issues with it. They did so….in responses that Steig didn’t see (the responses to review #3) because Steig was relieved of reviewer duty after submitting his third review. They didn’t incorporate those responses into the final draft, which Steig also didn’t see until later, when O’D sent him a requested copy. So from Steig’s POV, the issues were never addressed.
Yes, but Climategate.
Golf Charley, I’m with Keith in saying the jury’s still out on this. After reading Real Climate and Climate Audit and The Blackboard and Bart Verheggen, I just don’t know.
I see plausible scenarios where both ‘teams’ were acting in good faith, both responded in bad faith and of course, the scenarios being touted by each side now, where they were the good guy and the other was the villain.
The soap opera will clear up over time. The important thing to note is that it seems clear that there is not enough data to say that the Western part of Antarctica is warming more than a fraction, unlike the Peninsula that has been warming for decades. IIRC, people were predicting the melt on the Peninsula back in the 30s, for basically mechanical reasons unrelated to climate.
PDA, in deference to our host, I will content myself with saying that I’m glad nobody has been accused of deleting emails or leaning on journal editors or replacing one data seris with another.
So far at least, the Steig Trick or O’Donnell Gate is, from what I understand, not unusual in academic infighting around publications.
But Climategate. You don’t seem to understand that because Climategate was real and serious, it has predisposed the opinions of many to doubt proclamations from the Hockey Team, as they labeled themselves in a more innocent era.
But Climategate.
It’s still not entirely clear what this dust-up is about, exactly. As always, there are multiple intertwined story lines. Some science-themed and some personal.
As always, there are aggrieved principals, and choruses itching for a fight.
And as always, from the start, there are wildly different interpretations put on events by The Team and The Tribe (as Ron Broberg puts it).
Here is one view of part of the Science piece of the saga, describing the peer-review process. It may be more than you want to know.
Steig 09 performed an analysis of combined weather-station and satellite records, and concluded that (a) the Antarctic Peninsula is warming a whole lot, (b) West Antarctica is warming a lot, and (c) the whole of Antarctica is experiencing statistically-significant warming. This paper got the cover of Nature magazine.
O’Donnell et al blogged in late 2009 that Steig 09’s methods were mathematically wrong. Steig disagreed, and said, “well then write a peer-reviewed rebuttal instead of blog-whining.” O’Donnell et al then wrote up their analysis and submitted a manuscript to the Journal of Climate Science. The editor, Dr. Broccoli, sent the MS out to three reviewers: C, B, and A (Steig).
C looked carefully and liked the paper overall, but had reservations. B took a more cursory look, and also liked it overall, with reservations. A was extremely critical, and wrote a very long, unfocused review finding fault on dozens of points. It’s obvious to the reader that A is almost certainly one of the authors of the paper that O’Donnell et al are critiquing. More here.
O’Donnell et al extensively reworked and rewrote their MS. They largely satisfied C and B. A remained extremely critical, and wrote a second fault-finding review of unusual length.
O’Donnell et al again extensively reworked their MS. C and B were apparently satisfied. A remained unsatisfied on his major points, and wrote a third lengthy review. The editor then brought in reviewer D. D liked the paper and recommended publication. The editor had O’Donnell et al make certain added changes, and then accepted the MS for publication.
(It became clear this afternoon that the editor didn’t share the final, accepted version with reviewer A (Steig).)
A few days ago, Steig posted a long critique of O’Donnell et al on RealClimate with a patronizing tone (IMO). Some of Steig’s facts seemed a bit off, to one who had kind-of followed the story. In any even, O’Donnell evidently got angry, and fired back with a blistering post at ClimateAudit, breaking Steig’s confidence to reveal that Steig was Reviewer A (which was no big surprise), and claiming that Steig was a hypocrite — that as Reviewer A, he had asked for certain changes in the manuscript, and that he went on to fault those changes once he put on his RealClimate-blogger hat.
Steig’s rebuttal took the form of his follow-up post of this afternoon.
Forgotten in the bushes is the original claim that prompted O’Donnell et al to write their MS: that the main method of temperature reconstruction used in Steig 09 was mathematically wrong.
Is it?
There’s some evidence that this may be so. O’Donnell has presented a set of images that suggest that the main algorithm used in Steig 09 has problems. The first frames show that Steig’s method yields a picture of Antarctic temperatures that is sensitive to the addition of a synthetic +0.2 C or -0.2 C trend to the weather stations on the Antarctic Peninsula. Basically, this is what you’d expect (though the artificial rise is smudged over a wide area). However, the following frames show that the algorithm is insensitive to addition of such trends to the only two weather stations on the West Antarctic, Russkaya and Byrd. That’s counter-intuitive, and could well be a sign that something is quite wrong with Steig’s method (as implemented by O’Donnell).
Steig has not yet addressed that point. Presumably, like O’Donnell, he’s been busy. Or, he may not want to. We’ll see, over the next few days.
Steven Sullivan, # 17,
So I need to change the customer requested dry macaroni to the customer requested a replacement for the twice baked potato and in the review said “the filet should have been accompanied by a twice baked potato” <g>
A question: Do we know that O’Donnell was unaware that Steig had not seen their response to his third review or did he honestly believe that he had already answered Steig as a reviewer and Steig was “pretending” it hadn’t happened? In other word is this all a simple misunderstanding caused by miscommunication between two authors whose attachments to their papers made them lose perspective?
You want a 10 second speech? Try blogs.discovermagazine.com/2011/02/05/the-braying-wolves/#comment-42597
Every scientific disagreement has to become personalized. Every point of contention driven beyond mere contention. The Hockey Stick grew a curve. Mann still has a job. Climate-gate came and went. Jones still has a job. JO10 came out and the Antarctic is still warming. All this “perfidy” is exposed and adjustments are made, conclusions revised, and the science just keeps rolling on. This is unacceptable to certain Tribalists. They want blood. They don’t understand that is the way that science works 99% of the time. Incremental changes. They want to stop the hands of time, to have a reckoning. Assimilation of their technical critiques is offense to their sense of grandiosity; They will not be assimilated until their unique genius is recognized.
Shrug.
Resistance is futile.
RickA,
As can be seen by reading the material posted by O’Donnell, what really happened was:
O’Donnell et. al. note that irigde is better and produces similar results.
Steig, as reviewer, says, if it’s better and shows the same results, why don’t you include it in your paper.
It is simply dishonest to continue to claim that Steig suggested this method when the documents clearly show that it is false. Look for yourself. The first mention of this technique is made by the authors, not the reviewer.
the Hockey Team, as they labeled themselves in a more innocent era.
Wow Tom. Do you call it the “Democrat Party,” too?
Rather, as demonstrated in IPCC(2001) [see this comparison here] and numerous additional studies since, it is what is perhaps more aptly termed the “Hockey Team””“that is, the multiple independent reconstructions and model simulations that now indicate essentially the same pattern of hemispheric mean temperature variation in past centuries, that support a “Hockey Stick” description of past temperature changes.
http://www.realclimate.org/index.php/archives/2005/01/peer-review-ii/
10 seconds up:
Steig writes a crappy paper, Ryan writes a better one to prove it, Steig won’t admit his paper sucks & trashes Ryan’s in public, Ryan trashes Steig back and tosses in dirty laundry pictures.
10 seconds down:
Scientists can, yea must disagree, but when you’re wrong you need to cop to it, otherwise the guy/gal that got it right will get pissed & vindictive. I’m sure a journalist can relate……
As always, Gabriel gets it right, Games Without Frontiers:
“Andre has a red flag, Chiang Ching’s is blue
They all have hills to fly them on except for Lin Tai Yu
Dressing up in costumes, playing silly games
Hiding out in tree-tops shouting out rude names
-Whistling tunes we hide in the dunes by the seaside
-Whistling tunes we piss on the goons in the jungle
It’s a knockout
If looks could kill they probably will
In games without frontiers-wars without tears”
tonylurker
You are repeating the same thing from Bart’s place.
Where does Ryan claim, that Eric suggested ‘iridge’, denovo?
Ryan clearly states iridge was favored by Eric, in the context of Eric’s problems with TTLS.
Keith, at the 30000 ft level it is about antarctic warming. The warmistas were concerned throughout the early 2000’s, that even though they claimed the world was warming, Antarctica was cooling. So Steig gets his paper published, with a Nature cover, that says the west Antarctica warming was spreading over the rest of the continent. That was their bombshell. O’Donnell comes along and with much better math and statistics, proves that Steig not only exaggerated the data (via his inferior stats), but that even the data he did report was random and pure luck. You got the rest from above, basically Steig, who was conflicted, held up the O’Donnell paper for over a year.
#3
Gryposaurus lays out the timeline at Bart’s: see his comment at February 9, 2011 at 21:10 et seq.
The timeline I provided at Bart’s was only about the controversy regarding the accusations that Steig had “rigged” the review to suit his own purposes. Not who is right or wrong about the science. I tried to get in as much about the iridge without getting technical. I added more paragraphs a few posts later for ODonnell, to get his response to Steig’s final review. At first I thought was unnecessary, but it was unfair of me, so I added it. But now, it turns to be irrelevant to this story (very relevant to the science tho) because Steig never saw it, nor did he see the final revision, so the lie that Steig “pretended” to not know about the paper by publicly asking for a pre-print is also…well…a lie. Once again we are shown why it so difficult to get any message across to anyone. The wolves have been out lately.
From Steig’s piece
Many of my colleagues have warned me many times not to trust the good intentions of O’Donnell, Condon, and McIntyre. I have ignored them, evidently to my peril. But you know what has given me the most pause? The fact that a number of my colleagues and many otherwise intelligent-seeming people still seem to treat these guys as legitimate, honest commenters, whose words have equal weight with, say, those of Susan Solomon or J. Michael Wallace, or, for that matter, Gavin Schmidt or Mike Mann or myself. As a reporter wrote to me today “it’s simply impossible for a lay observer to make a judgement on his/her own.” Really?!
Perhaps there is a silver lining here. Perhaps the utter silliness of the shrill accusations that O’Donnell made against me “” based on a version of the facts, in his head, that are demonstrably and unequivocally false, coupled with the fact that he then retracted them (or at least has promised to do so), will help more people see what the steadily growing list of other scientists who’ve been accused by McIntyre and his associates of plagiarism, dishonesty, data manipulation, fraud, deceit, and duplicity have been telling me for years: these people are willing to say anything, regardless of the cost to others’ reputations and to the progress of legitimate science, to advance their paranoid worldview.
Hmmm. There appears to be story here that’s being missed. This will always be ugly. And the other side has it’s thorns, But Trenberth, But Schmidt, But Steig. Remember But Mann, But Santer, But Jones. Sometimes There’s stories that are so in your face that you don’t even need to fish for them.
Seriously, this latest episode was so ugly and one-sided (in terms of not hearing a response from the other side)
Why is that a surprise? One side is full of smear merchants and those eager to quickly echo smear without a shred of questioning. The science side is usually a little too busy to quickly deal with every bit of daily soap opera nonsense that originates in the garbagesphere. Those who support the science side tend to not be so quick to jump to conclusions either.
Or in other words, the famous quote
A lie can make it half way around the world before the truth has time to put its boots on.
KK,
Do you think abuse of peer-review matters in science?
Reactions to this are complicated by the history around the publication process. Notably, the O10 paper seems to be at least in part a response to various IPCC-side remarks about how their unserious responses to criticism were justified because the criticisms hadn’t been put through the academic journal system; given that, any <a “http://climateaudit.org/2011/02/07/eric-steigs-trick/#comment-253908″>seemingly unserious or unreasonable response to O10-related stuff</a> becomes especially aggravating. And given the McIntyre-o-sphere laundry list of complaints about past reviewer mischief, they may be even more than usually sensitive to things like a Team reviewer going to town on a paper that other reviewers seemed satisfied with. Also, some of the irritation about the quality of the S9 paper follows from how Nature chose to put it on their cover: as in the earlier Hockey Stick controversy, unless the featured paper is a very solid technical result, that extremely visible editorial decision is disturbing and extremely annoying.
There’s also free-floating bad blood like Steig writing <a href=”http://www.realclimate.org/index.php/archives/2010/06/what-do-climate-scientists-think/comment-page-2/#comment-178909”>… “c) that we consider it dishonest or disingenuous with respect to the science. Pielke Jr, Blackboard, and ClimateAudit all fall squarely into the latter category.–eric”</a>, and Steig being mad about how, as SteveM puts it, <a href=”http://climateaudit.org/2011/02/07/eric-steigs-trick/”>”some of Ryan’s language, including the original title, breached blog policies and has been edited accordingly”</a> — where the new policy-compatible title is “Steig’s Trick”:-|.
I think Jeff Id not being let through at RC to post a defence of his own paper helped get this ball rolling somewhat.
A point that has occured to me is this. If Dr. Steig hadn’t seen the final draft then his intial post at Realclimate was on the third draft.
How do you, in all good conscience, write a blog post about a paper that you haven’t seen the final draft of?
Wouldn’t it be a rather basic bit of fact checking to ensure that the paper you writing a critique of is in fact the one that is published?
KK, if you as a journo were going to write an article about a report being prepared for the Government, wouldn’t you check that you are writing about the report as presented and not some intermediate draft?
I would really like to hear from some published researchers on the matter of Dr. Steig reviewing a paper that (perhaps) rebuts his own. Is this normal proceedure? This smells very strongly of “Conflict of Interest” to someone from outside the published scientific field.
Stu, I do think one of the more fascinating aspects of this issue is RealClimate’s moderation decisions in relation to it. I think Amac’s experience (as documented over at Lucia’s blog) is the most incredible.
RealClimate edited out part of Amac’s post without giving any indication of the edit. When Amac submitted a comment objecting to this, RealClimate deleted it.
I don’t get why they would do something like this. It can only make them look dishonest. If they didn’t want the removed text to appear, they could have deleted the comment or even just noted the edit. Without doing either, they create a misrepresentation which serves no legitimate purpose.
There was I though a very sensible comment at RC, and Eric has responded, in a slightly puzzling way, I could read it a number of different ways? Any thoughts?
http://www.realclimate.org/index.php/archives/2011/02/odonnellgate/comment-page-1/#comment-200181
I do have one point that I would like to raise, although I don’t know the whole story (personal emails “¦etc). I don’t mean it to be accusatory in any way so please don’t read anything into what I suggest.
This all started with the article here describing their paper, where a minor technical disagreement and (IMHO) a misunderstanding of what Eric had written and meant. This could largely have been avoided if O’Donnell et al had been contacted about this article and been given a chance to reply with a post of their own or within said article. Their grievances would have been dealt with privately before anything was published and it wouldn’t have gotten out of hand.
Let me try to rephrase this. If this was a paper published by someone else (Susan Solomon for example) and you had serious misgivings about it, would you invite her to reply to these objections? I think this is the lens that O’Donnell et al view this through and why they feel they are being treated so differently (and why they are quick with insults and accusations).
As I said this is not an accussation, I just wanted to stimulate some discussion.
[Response: I think if I had problems with a paper of Susan’s, I’d write a serious of blog posts criticizing her integrity. That seems to work pretty well. Do you think I’ve learned nothing from O’Donnell? …
… yes I am kidding.–eric]
Ultimately, what next…
from 60,000 feet, Anratica would appear not to be warming in the way that generated an alarmist Nature cover that went around the world..
whereas the paper showing that this previous result is almost entirely wrong did get published, but only in a much less prominent journal and without all the PR in the media..
If this were science, O’donnell and Steig would have very possibly worked together to produce a paper that improved on the previous one…
Yet, as this is politics now, the paper and front cover that were elevated to an icon to sell the AGW message to the media, public, and politicians, it all descends into a bitter fight. and the actual GOOD news gets forgotten.
@JohnB
“I would really like to hear from some published researchers on the matter of Dr. Steig reviewing a paper that (perhaps) rebuts his own. Is this normal proceedure? This smells very strongly of “Conflict of Interest” to someone from outside the published scientific field.”
I do not work in climate science, but yes, in biology this is COMPLETELY normal. I believe it is important to stress this. And this is really not a problem, because the editor is aware of the possible conflict of interest and if the editor believes the reviewer is being unfair, he can ignore his comments and follow the other reviewers, or (as is done here) assign an extra reviewer. As author, you can always, when submitting an article for publication, request that you don’t want a particular people to review the work, if you are afraid he would be biased. Such requests are most often granted. I am not sure, but from all this I gather that O’ Donnell did not make such a request, or we would have probably known. Personally, I have never made such a request, because I believe your paper becomes much stronger if your “worst” opponent reviews it. After all, the goal of the paper is to convince your opponents, isn’t it? Of course, as I said previously, it will sometimes happen that a reviewer abuses his power and in such cases the editor should (and most often does!) step in.
For these reasons I really think that Shub’s suggestions that peer-review is abused here is nonsense. Working scientists are really used to harsh criticisms and don’t have a problem with that. It becomes a problem when it gets personal, for example by accusing someone of dishonesty, such as O’Donell has done.
@John B
I forgot, but also in climate science this is normal (thankfully). For example, McIntyre reviewed the Wahl and Amman paper that refutes their work.
http://climateaudit.org/2006/03/28/letter-to-climatic-change/#comment-47365
As I said previously, I think that is only normal. But isn’t it strange that in such cases we never here any complaints about “conflict of interest)”?
@milanovic,
You are right, the peer review process can work successfully with regard to the first author reviewing the second, and up until last week it did work adequately with these guys (neither was probably thrilled withe the arrangement but they lived with it). At the core this is not about peer review.
The entire problem comes last week, when Steig posts “West Antarctica: Still Warming” at RealClimate. The tone is dismissive, his comments are from a superior intellect that has to deal with the childish mistakes of a well-meaning but misguided neophyte. This gets posted on a well-read site where Steig has moderation control, and when Jeff C writes to rebut the piece, his submission finds its way to the “Bore Hole”.
In the old days, when stuck on the “losing” side of the media battle, Ryan & the boys would gather at the corner bar to have a few beers & grouse about what a prick Steig is, but guess what? It’s not old media anymore, it’s new, and there are many sites available to Ryan & the boys where they can state their piece. And they do, with venom. The peer review stuff is just an “oh by the way” at the end, at the core this argument is about whether Steig’s approach and conclusions are valid. Steig does not engage Ryan on these points.
Steig provoked this fight and his only way to slink out now is to play the victim; he should not be allowed to do that.
Hi Keith,
It’s not that complicated. There are two parts:
1) The science — is Antarctica warming? Or is another key prediction of IPCC models refuted by observations? While still not settled, the appearance, if you read both sides, is that Steig et al. have failed to show real warming in Western Antarctica. See e.g. http://climateaudit.org/2011/02/08/coffin-meet-nail/. This dispute will doubtlessly continue, but it has all of the appearance of a comprehensive rout on this particular issue.
2) But perhaps the bigger issue, after Climategate of 2009, is the personal ethics of ‘Team’ scientists. Just a few weeks ago they were telling us this: http://www.realclimate.org/index.php/archives/2011/01/unforced-variations-jan-2011/. I listened with an open mind. I was told, as we all were, that we would all shortly see for ourselves that their moderation, which has often been called censorship, is and always has been purely a way of keeping threads scientific, civil, and on topic. A few weeks later Lucia has shown us — http://rankexploits.com/musings/2011/in-moderation-at-rc-yes-no-whatever/ — nothing has changed, and their moderation is indeed censorship, as nearly anyone even vaguely skeptical of climate change has always suspected.
And then of course, there’s the Team antics in manipulating the peer review. We’ve known for a while that O’Donnell et al. had to jump through extraordinary hoops in order to have this latest manuscript published. At the same time, as again Lucia has shown, Steig was making blog comments pretending not to know anything at all about the forthcoming O’Donnell paper. Now we find out that he was the very same reviewer who was trying to block publication of the manuscript.
After Climategate, I just have to ask, how come they don’t learn? Their inability to learn, to me, suggests a compulsive, shared control addiction.
I suppose you can shrug it off and say ‘big deal’. If so, I’d say your expectations are not very high. Perhaps after Climategate, that’s understandable.
Best regards, Alex
@MattB
Well, if found the tone of Steigs first post really moderate and constructive. When I read O Donnells post I was actually shocked. “There are not enough vulgar words in the English language to properly articulate my disgust at his blatant dishonesty and duplicity”
I am amazed how you can take issue with Steigs’ tone and defend the post of O Donnell.
“Steig does not engage Ryan on these points.”
Well, after being accused of lying, I can imagine he first wanted to sort that out. It was all over the blogosphere that he was lying, dishonest, weaseling etc.
@milanovic,
I will agree with you that Ryan’s post on Climate Audit went over the top.He was ticked off & should have moderated the tone, because now the discussion is all about what he said about Steig, and very little about whether Steig’s original paper had scientific merit and why Steig keeps defending it by trashing the Ryan paper on Realclimate.
I will disagree with you on Steig’s first RealClimate post being “moderate and constructive”. He writes dismissively on Ryan’s paper and then will not publish contravening points of view from the paper’s co-author. This is yellow journalism and cannot be defended in any forum, and it certainly should not be characterized as “moderate and constructive”. I am amazed that you would defend Steig’s approach.
@MattB
“He writes dismissively on Ryan’s paper “: here I disagree, for a scientific discussion, his tone is not very harsh. For a climate science discussion I would even say it is extremely moderate. I guess you have read reviews of McIntyre of scientific papers? Wouldn’t you agree that Steigs review is is definitely moderate in comparison!
But I was also disappointed in the moderation policy of RC. That was actually one of the reasons I at first felt that O Donnell may have had a point.
@milanovic,
I agree that I have heard harsher comments in the past than Steig made, but he did come across to me as condescending, and Steig did not make his arguments in a forum where there is give and take. It was done on his home site, where he had had the ability to censor the discussion and indeed he did use that censorship. Again, yellow journalism.
I would like to see them discuss the true scientific issues where both sides can lay out their case cleanly. It looks to me like Ryan will argue his case in an environment where there is a free flow of ideas ; we’ll see if Steig has any interest in venturing outside of “protected” sites.
John B,
As milanovic also said, yes, it is completely normal, or expected even, that authors whose paper is being critized are one of the reviewers.
As e.g. Pielke Sr wrote about Menne’s paper (via RC):
“I was quite surprised to learn that despite the central role of Anthony Watt’s analysis in the paper, he was not asked to be a referee of the paper. This is inappropriate and suggests the Editor did not provide a balanced review process. “
There is also Steve McIntyre complaining that Climatic Change breached their contract with him
by not sending him the final version of the Wahl and Amman (2006) paper.
And Steig responding at RC about a potential reply to O’Donnell:
So if it is now considered that Steig 09 was flawed, how did it pass peer review?
Or was it pal review?
Reading a bit more and looking a bit at the history, I think O’Donnell lost his temper in a ‘last straw’ manner, as has happened to me just in the comments sections of this and other weblogs.
Real Climate games the system in every way it can when it comes to engaging with opponents. One of the worst is when in the middle of a discussion in comments, they ‘disappear’ a comment. Very frustrating, and something other consensus bloggers have learned to take advantage of, just so they can feign horror when the aggrieved party loses his (why is it always ‘his’?) temper.
O’Donnell probably should have stepped away from the keyboard. But he was provoked.
Real Climate won the battle–we’re talking about the food fight, not the findings. But that will pass, and their behaviour gets a bit more obvious with every contretemps.
Tom,
You’re not exactly a dispassionate observer when it comes to Real Climate and the scientists involved.
But I would agree that their commenting systemleaves something to be desired. Anyway, I’ll have more to say in a bit, drawing (perhaps) some useful comparisons to the magazine journalism editorial process.
milanovic and Bart. Thanks for the information.
To a degree I suspected it was relatively normal simply because the number of qualified reviewers in any given field must be limited and so they would of necessity finish up reviewing each others papers.
I must add though that looking in from the outside it is still a concern. Most industries dealing with reports have very stringent rules concerning “Conflict of Interest” to prevent abuse. And often there are legal penalties for such abuse.
Science does however seem to operate from a point of perceived ethical superiority. It’s like science is saying “We don’t need these strict rules that others do because we are scientists”. “Yes, there might be C of I, but our journal editors are very smart and would put a stop to that sort of thing”.
Why are editors etc of journals so much smarter than everybody else?
I’m not sure that I’m explaining myself well here and I’m not trying to cause offence. I’m certain now though that this is an area where those from outside science distrust the system of science. I’m not out to carp on about C of I but it makes a good example.
Outside science, C of I must be revealed. The law says so. The laws are written to prevent abuse and investigations and court cases might follow. Yet inside science all this can be safely replaced by the opinion of an editor.
I have to view such a state of affairs with concern.
It appears that the editor relieved Steig of his revieweing duty after his third review.
I wonder why?
John,
Examples of conflict of interest would be if the reviewer has recently collaborated with the author on the same or a similar project, or (much more vague and harder to prove or disprove) if the reviewer is currently working on the same topic and therefore has a vested interest in delaying the article, because (s)he wants to be the first one to publish.
With criticizing another article, it makes sense to grant the author of the critiquend article a right of reply (or in this and most cases, right of review). Of course the editor needs to be aware of the position of this reviewer as the one being critiqued and weigh the review accordingly with other reviews from more disinterested parties.
kdf33 “It appears that the editor relieved Steig of his revieweing duty after his third review.”
From reading the correspondence after the 3rd round of review JoC asked O’Donnell to respond only to reviewer A (Steig). The response basically disagreed with reviewer A and 3 weeks later a decision was sent from JoC saying they were satisfied with the response and only asked for minor changes in the abstract. So Steig wasn’t relieved of his reviewer duties, after 3 rounds of review JoC decided his objections had been addressed.
JimR,
Fine by me, but how does that square with #17 above.
Andrew Revkin has a good post up on the controversy, that is part overview, part meta-take home lessons.
#17 points out that reviewer Steig never saw the 4th (final) submission of the manuscript. Editor Broccoli directed the O10 authors to respond to the points Steig made in his review #3, not necessarily to change their manuscript (3rd version) to attempt to satisfy Steig’s objections.
“MrPete” in the RealClimate comments links to a PDF he made that highlights the differences between MSv3 (which Steig critiqued) and MSv4 (which Broccoli never shared with Steig). The differences are quite minor. In other words, unsurprisingly, O’Donnell chose to write a cover letter to Broccoli, defending their choices against Steig’s remaining objections, and accompanying the letter with a MS that had only modest revisions.
So, as Reviewer A, Steig had seen and critiqued a nearly-final version of the MS. But he wouldn’t have known that for sure.
In this regard, Broccoli’s conduct seems unexceptional. Steig had volunteered to review the MS and had made his contributions. In the end, Broccoli allowed O10 to be accepted without addressing all of Steig’s objections (which would have been near-impossible, IMO). It’s not unusual for reviewers not to see the final version of a paper pre-publication, in my experience. After acceptance, the editor’s focus shifts away from the reviewers; they’ve done their part.
I don’t think it’s fair to say that Steig was “relieved of” his duties as reviewer. He fulfilled them. Broccoli wasn’t obliged to act on all of Steig’s advice.
kdf33, It doesn’t square with #17 as that post assumes Steig was relieved of reviewer duties and didn’t see changes. From reading #17 I’m not sure he read the reviews, responses and correspondence from JoC. For example there were not responses reviewer A (Steig) didn’t see as the last response by O’Donnell to reviewer A disagreed and said repeatedly “We do not intend to change the manuscript”. That response was found to be satisfactory to JoC. The differences between what Steig saw as reviewer A in the 3rd round and the final draft were minor.
It also explains why Steig requested a final version of the paper, and why some implied that this was sneaky/duplicitous.
Steig assumed that there would be significant changes to the final version that he had not seen.
O’Donnell thought Steig asking for a final version was redundant as unbeknown to Steig there were no further significant revisions.
O’Donnell and others assumed Steig’s request was a ploy of some sort, possibly to obscure his role in the review process.
What’s most disheartening about this whole thing is how utterly unsurprising it was. What started out as initial courtesy gave way in fairly short order to shaky accusations of impropriety by a well-respected researcher.
And you know what? If O’Donnell wasn’t so steeped in the incessant, poisonous, conspiratorial innuendo of places like CA, tAV, it actually sounds like he could have had a productive back and forth Steig et al.
tb, “Steig assumed that there would be significant changes to the final version that he had not seen.”
Is that just an assumption on your part? I’m curious as reading the response to Steig’s 3rd review it’s hard to understand how Steig would expect significant changes when his criticisms were met with disagreement and statements the manuscript would not be changed. I’m sure Steig would be curious of the final form of the paper, but assumed there would be be significant changes?
#59, And you know what else? If The Hockey Team had not consistently engaged in a pattern of behaviour that aroused such suspicions, O’Donnell might not have been so touchy.
This seems to be a big omelette, with plenty of egg to cover multiple faces.
Raypierre H. has a response on RC today noting that he too has gotten reviews that were of as long as the one Eric wrote, and it wasn’t even on a hot-button topic.
http://www.realclimate.org/?comments_popup=6764#comment-200350
And of course it is not extraordinary to have an author who is targeted in a critical paper, be *among* the reviewers of that paper, as Steig was….nor would one be surprised if that reviewer returned a particularly detailed critique. You think that’s all ‘abuse of the peer review system’ Shub?
It’s too bad so many members of the ‘Tribe’ appear to have little experience of the actual processes of scientific writing , review, and publication, but they shouldn’t use their inexperience as a cudgel against those who do.
@60 JimR:
Agreed. Should have written “might” rather than “would”.
@61 Tom Fuller:
This is something that I think many people don’t seem to get: there is a difference between doing science and statistical grammar-Nazi-ing. Places like CA and WUWT seek to conflate the two, but they’re different. Some people, like Steve McIntyre, appear to be more than content spending their time attacking a small group of researchers repeatedly over their statistical choices. Fair enough. That’s certainly his right! It’s not science, and it’s never going to be science.
O’Donnell seemed like he was interested in not only criticizing Steig et al., but in actually trying to help figure out what is really going on in terms of Antarctic trends*. Steig seemed to be happy to take him up on that, and acknowledged that in some areas, O’Donnell’s analysis was probably better than his own. However, even independent climate people like Hyubers agreed with Steig that it probably wasn’t helpful to just dump the data that O’Donnell did. There was room for, if not collaboration, an exchange on how to best incorporate the good from O’Donnell’s paper while making use of some of the data it threw out in order to even better describe the behavior of the real world.
That’s probably not going to happen, at least not with Steig and O’Donnell. And that is solely due to the atmosphere of hostility, disrespect, and distrust created and stoked by people like McIntyre, Watts, et al. If O’Donnell was always content to be another statistical grammar Nazi, I guess it’s not such a big deal.
But if he was interested in contributing to science, this is unequivocally his loss.
*Compare this to McIntyre’s utter refusal to actually perform a multi-proxy reconstruction of his own.
#62, Raypierre H makes some good points in the linked Response at RC (I’ll respond here as effective today I’m blocked from submitting comments over there.) He is also somewhat disingenuous, IMO. Yes, lengthy reviews can be a boon to authors. I find it very hard to see Steig/A 1, 2, or 3 as falling into the “more helpful than not” category. Recall that O10 was accepted without satisfying Steig/A. In my opinion, occasional quotable kind remarks notwithstanding, Steig/A would never have agreed to acceptance of O10 in a recognizable form.
IMO, it’s not Steig/A’s fault that his conflict-of-interest as a reviewer wasn’t handled better — that was Editor Broccoli’s remit. “No good deed goes unpunished” as a summary of either Steig’s or O’Donnell’s actions — I hope few people will sign on to that hyperpartisan sloganeering.
I do have experience in scientific writing, review, and publication (which you can believe or not, given that I’m pseudonymous). I remarked further on the O10 peer-review process at Lucia’s (Comment 68919).
JohnB: A point that has occured to me is this. If Dr. Steig hadn’t seen the final draft then his intial post at Realclimate was on the third draft.”
No. Steig’s post was based on a preprint of the published version given him by O’Donnell at Steig’s request, when they were still being collegial. It was the first time he’d seen it; nor had he ever seen the responses to his third round critiques, which addressed several of them, nor were these responses incorporated into the final draft. Hence the confusion.
#55kdk, if my use of the term ‘relieved of duty’ in #17 confuses you so , feel free to substitute phrase sums this up for you: ‘Steig was a reviewer for the first three drafts, but was not asked to review the final version, nor, apparently, was he sent O’Donnell et al.’s response to his third review”
#57 Amac
Nice ‘praisee’, but actually Steig expressed amazement on RC comments at the possibility that O10’s fourth draft wasn’t reviewed by someone. He was apparently under the assumption that it had been. Hence more confusion. Yes, the editors apparently thought the work of the reviewers was done, but Steig did not.
http://www.realclimate.org/?comments_popup=6764#comment-200182
Were there reviews of version 4?
The reason I’m asking is that these do not appear to be in the archive along with all the other reviews that O’Donnell has placed on Climateaudit.org.
Surely he is not allowed to post all that material anyway?
[Response: I don’t know. If there weren’t then I’m stunned.–eric]
Comment by Deep Climate “” 9 Feb 2011 @ 5:20 PM
#64 Amac:
Yes, you already referred to review A.1 as ‘unfocused’ in your very first attempt at a summary on this thread. So yes, it’s not surprising you find it ‘very hard’ to consider 1,2, 3 three ‘helpful’. Your stance is clear, thanks, and doubtless influences your firm belief that Steig would ‘never’ have agreed to have O10 in a ‘recognizable’ (!) form.
Indeed, O10 was published without **fully** satisfying Steig/A. Yet Steig appears to think that the review process, presumably including his reviews, made O10 better. Changes from v1 to v2 to v3 either bear that out or not. Are you saying they don’t?
Steven Sullivan,
#66, What’s ‘praisee’?
#67, I haven’t read v1, v2, v3, and final — too much of a time sink. I take from all parties’ comments that the MS was much improved between first submission and final form. Do you have an opinion as to the best version?
AMac, your comment over at Lucia’s is the best mid-game summary of this that I have read. Well done, and I hope you post it over here (I just pasted chunks of it in over at Bart’s).
O’Donnell thought Steig asking for a final version was redundant as unbeknown to Steig there were no further significant revisions.
O’Donnell and others assumed Steig’s request was a ploy of some sort, possibly to obscure his role in the review process.
That’s how I read it as well, but it makes little sense, since he already knew Steig was a reviewer at that point. Somehow Steig, who already revealed to O’Donnell when asked that he was a reviewer (on the condition that O’Donnell would keep that private, which he clearly didn’t honor), was trying to hide his role by asking for a copy that O’Donnell assumed he already had? What? Even if it didn’t occur to O’Donnell that Steig never saw the final version (one of his many false assumptions), believing some sort of dishonest intentions is utterly irrational. While one can dismiss this as the atmosphere of the McIntyre crowd of conspiracy nuts turning him into a paranoid fool, he’s an adult I believe, and should be able to think for himself.
“Steig was a reviewer for the first three drafts, but was not asked to review the final version, nor, apparently, was he sent O’Donnell et al.’s response to his third review”
Oh, I get it, he was relieved of duty. Got it. Thanks.
Sounds like the editor had enough and either got a different reviewer or just let the disagreement stand unresolved.
#68, “What’s “˜praisee’?”
A minor joke (not at your expense). A Lisbon ‘reconciliation conference’ reference. It’s ‘precis’ in Tallbloke-speak. And if you don’t know who or what Tallbloke is, don’t worry about it, you absolutely aren’t missing anything.
As for the drafts, like you, I’ve not read through them, though I’m hearing that there was much improvement from 1 to 3, and little change from 3 to 4. Was wondering therefore why you assume Steig’s input was not helpful. That some of his recommendations were not taken up, doesn’t mean NONE of them were.
#71 kdk3, I refuse to take responsibility for what one such as yourself does or does not ‘get’. If suggest you read for yourself what the parties involved have written, on RC and CA.
#72 Steven Sullivan, thanks for clarifications. In the RC Comment thread, “MrPete” compares v3 and v4, finding very little difference. RyanO says the same, at Lucia’s. Thus, “much improvement v1 -> v3” is about the same as “much improvement v1 -> v4”.
As to whether Steig/A was helpful on balance, I doubt that will be resolved any time soon. RyanO has said that Steig’s reviews were helpful on some matters, which rules out “completely unhelpful.” At RC, Steig is at pains to say how helpful he meant to be and was. At Lucia’s again, Nick Stokes (whose field is close to O10’s subject matter) also thinks that Steig’s reviews were helpful. All points in Steig’s favor.
Reading through the reviews and imagining myself as an author, that was not my impression. The sense I got (FWIW) was that Steig/A was sprinkling encouraging, general words among some very negative specific commentary, and setting up a series of requirements (or suggestions). Some of them are acknowledged by RyanO as improving the MS (and some of these were similar to input of C and B), so not much doubt there. Others of Steig/A’s tasks struck me as being obstacles to acceptance. Indeed, a reviewer can weigh in for a rejection without explicitly proffering that advice by advising, “Acceptable, if…” followed by a series of onerous conditions. In the end, Broccoli accepted the paper after RyanO indicated that he would not attempt to meet Steig/A’s remaining demands.
At RC, Steig says he didn’t intend to do that. I can certainly see how the O10 manuscript’s authors might have come to believe that could have been on his agenda.
Scientists have a reputation of being smart, but often somewhat deficient in social skills. That may have supplied the spark to a situation where a lot of tinder was already laying about.
Welcome to the 21st Century. Please choose immediately–pseudo autism that allows you to work or pseudo-attention deficit disorder that allows you to cope. Choose only one. You must choose one. You will not be allowed to change.
Provigil and Xanax, or coffee and beer. For every problem industrial society creates, it also provides a solution!
@65 Steven Sullivan.
Thanks for that clarification.
I’m reaching the point that this whole affair can only be described as “The biggest balls up since the elephant fell over at the zoo.”
@73
Are you sane?
Everyone seems to know what Steig, O’Donnel and the editor were thinking at various stages, and that ‘knowing’ tends involve projecting motivations onto them so there actions are interpreted in the most hagiographic light of the protagonist they favor and all with little self-awareness they are doing this.
dorlomin–
Many know what Steig, O’Donell and the editor wrote, in what order these were written and the fora in which they were written and are evaluating, diagnosing and judging various actors actions and characters in this context.
I happen to judge straight-forward hot-heads who sometimes make mistake to double-dealing people who buttress their case by presenting half-truths, and censoring adverse fact. I don’t need to guess what anyone is thinking to detect when a case is made by half-truths and when someone was just a hot-head. I don’t think anyone else read minds to detect this either.
They just need to be aware of more facts than presented by the person crafting story by half-truth.
O’Donnell gave his word to Steig that he’d protect his anonymity, then broke it. O’Donnell’s been criticized by Steig and his allies for that. Justly, IMO.
O’Donnell got some key facts wrong, misinterpreted others, and quickly posted without checking his assumptions. Just criticism for that as well.
O’Donnell wrote blog posts in anger. That may not be a crime, but it was surely a mistake (Talleyrand paraphrase).
O’Donnell’s release of reviews and correspondence seems to have caused embarrassment and headaches for both him and Steig. For scientifically-literate laypeople who are concerned with the state of peer-review in climate science, the files offer an important behind-the-scenes view.
Steig has said that O’Donnell is preparing a public apology for certain things; this hasn’t been disputed. We’ll see how that turns out.
AMAC,
“The sense I got (FWIW) was that Steig/A was sprinkling encouraging, general words among some very negative specific commentary, and setting up a series of requirements (or suggestions). Some of them are acknowledged by RyanO as improving the MS (and some of these were similar to input of C and B), so not much doubt there. Others of Steig/A’s tasks struck me as being obstacles to acceptance.”
Can you expand on this? Obstacles to acceptance can mean different things. You are either questioning Steig’s ethics, or you are saying that Steig was doing his job as a reviewer, or perhaps something else. Any substantive complaint about the paper is going to be ‘an obstacle’ in any review. The point of peer review is not to assist ODonnell in getting a publishing credit. His anger or deluded nature (whatever his deal is) of what he thought Steig was ‘doing’ is irrelevant. That is between the editors and ODonnell. The system works, it is not up to science to set up a system where everyone gets what they want, or to make it appear ‘fair’ to people who have no idea what the process is about. How about we explain how the process works, instead of assuming the worst. Or don’t submit to journals that care what they print.
What would you, or anyone else thin is a good way to handle peer-review in this situation? Keep in mind, journals that are looking or the best possible science want the anonymous factor, to not scare away reviewers, and get the opinion of the people with the most knowledge on the subject, which may include the ‘conflict’ and in fact, usually does. There seems to be this idea that because O”Donnell freaked out, that there is now a problem with peer-review, thanks to Revkin’s article. This is just patently false, unless otherwise displayed. Unless there is a better system at getting to the truth, please someone let Revkin know that science existed before auditors decided it wans’t good enough to make them deal with their own paranoid fantasies.
Spin on the peer-review system is spin.
There are a number of interesting comments over at the Dot Earth thread. For the moment, I’ll just point out these two.
As Louis Derry says,
“It is common for a submission that critiques previous work to be sent to the author of the critiqued work for review. That emphatically does NOT mean the reviewer has veto power. It means that his/her opinion is worth having. Such a choice is usually balanced by reviewers that editors believe are reasonably independent, and the review of the critiqued is weighted accordingly. Suggestions that asking Steig to review O’Donnell was somehow unethical are utterly without support in normal scientific practice. Obviously, Steig did not have veto power over O’Donnell’s paper.”
All true, but Steve Mosher makes a valid observation his comment about the contentious history between Steig and O’Donnell and notes:
“Given the bad blood before the paper was written it seems that the editor did Steig a disservice by putting him in this no win situation.”
The question is, was the editor aware of the “bad blood” and even if so, does the previous “bad blood” matter, given Derry’s points about Steig just being one reviewer and his review being balanced out by other reviews?
grypo,
I offered my opinion at #21 and in the linked Blackboard comment. I see no evidence of bad faith in Steig’s reviews (caveat, I can’t follow the math).
The problem of Steig’s obvious conflict-of-interest doesn’t like with Steig, IMO. He was asked to volunteer as a reviewer, and he accepted the chore. I think editor Broccoli and the journal didn’t do an optimal job of managing Steig’s CoI. B, C, and D had similar views of O10, broadly speaking. A/Steig was the outlier, with his vehement objections.
Surely this situation isn’t an outlier, from the editor’s point of view. Author X writes a paper, author Y sees big problems and submits a MS claiming, “X approached this wrong, the issue should be handled as follows…” As an editor, I would certainly solicit X’s view of Y’s MS — who better to point out flaws? But I wouldn’t consider X’s review to be CoI free — it’s the conflict that makes X’s insights valuable.
As I stated earlier, I am overall encouraged by this peek at the machinery of peer review in climate science. Surely it is inconsistent with the cries of “conspiracy” that arise at places like WUWT.
Without more information we can’t know, but I suspect that Steig ’09 got an unduly easy ride in getting into Nature and on its cover. A more O10-style of review might have served all parties better.
Aargh. The problem of Steig’s obvious conflict-of-interest doesn’t lie with Steig…
Keith’s and Steve Mosher’s remarks supra are worth considering.
Thanks AMAC.
My anger wasn’t directed at you, it’s with the unqualified opinion of Revkin that
‘The exchanges between Steig and O’Donnell do raise questions about peer review, given that Steig has said he was an early anonymous reviewer on the draft O’Donnell paper challenging his own analysis’
“I suspect that Steig ’09 got an unduly easy ride in getting into Nature and on its cover. A more O10-style of review might have served all parties better”
Whether this is true or not, I think it’s important to realize that Steig’s work was the first and ‘first’ works usually changed over time. There was no O10 for the Nature to turn to. I highly doubt Nature would put easily refuted work on their cover if they had a chance to know of anyone who could review it more in depth.
“The problem of Steig’s obvious conflict-of-interest doesn lie with Steig…”
I sorta agree, but I’ll give you a similar (but by no mean perfect) analogy:
A few months ago, Trenberth reviewed RPJ’s book for a major science journal (can’t remember if it was Nature or Science). That struck me as an obvious conflict of interest, given their testy history (and the fact that Pielke Jr has been critical of Trenberth in the past).
In normal book reviewing world, that sort of thing doesn’t usually happen.
If the editor of the book review section of the journal wasn’t aware of the past history between Trenberth and Pielke jr, it was up to Trenberth to acknowledge it (he should have recused himself).
So I see a similar principle here. In good faith, it seems to me that Steig hopefully divulged the past history between him and O’donnell to the editor (if editor wasn’t aware). But then again, it’s not like Steig had veto power or was the only reviewer, so like I said, not a perfect analogy.
If the editor did know about the ‘bad blood’ he obviously thought that he would be able to detect any ‘conflict’. The editor was right to ask either way. It’s his job to get the best comments he can, and to be able to determine if the comments improve the work. Otherwise, he’s done a disservice to science and his magazine.
“It’s his job to get the best comments he can, and to be able to determine if the comments improve the work. Otherwise, he’s done a disservice to science and his magazine.”
I agree with you there, which is why my analogy is not perfect at all. (Whereas the book review editor at the journal could easily have found someone else–and just as knowledgeable as Trenberth– to review The Climate Fix.)
ODonnell’s apology:
“Steig informed me by email that he had not seen our Response to his Third Review, as I had previously assumed. I apologize for my misunderstanding on this point, which was, however, incidental to the major concerns expressed in my post.”
And
“In any event, Steig knew or ought to have known that our response must have satisfied the editor of Journal of Climate and should have familiarized himself with our response before condemning the method that he had previously encouraged. Had Steig informed me that he had not seen a copy of our Response to his Third Review, I would have been delighted to send it to him. Instead, he chose to publicly disparage our paper using arguments that were both irrelevant and satisfactorily addressed ““ which was, unfortunately, no different than the tactic he used during review.”
John Nielsen-Gammon at Stoat:
“Announcing the identity of the anonymous reviewer was wrong in and of itself. The seriousness of the offense deepens to the extent that the author also reveals some of the content of the review. Revealing the identity of the reviewer while simultaneously publishing the complete content of the reviews makes this particular ethical violation as bad as possible.”
“But then again, it’s not like Steig had veto power or was the only reviewer, so like I said, not a perfect analogy.”
That’s not really how it played out, though.
If it was appropriate for Steig to be a reviewer (which is itself hotly debated), this should have been subject to his fully disclosing his relationship with the authors to the journal editor. With all that under consideration, and with due consideration given to the fact that the paper under review was a direct challenge to the reviewer’s own work, the JE should have given proportionately measured consideration to Steig’s views on the paper under review, with this always at the forefront of his mind.
However, as can be seen from the disparity between suggestions made by reviewers B and C, and the three(!!!) obstructive reviews from Steig, the months of delays resulting, and hoops that O’Donnell et al were forced to jump through materially in order to maintain the journal’s interest in publishing them at all, there can surely be no question that Reviewer A was given disproportionately great influence for far too long. Bear in mind that the paper was not, in the end, published as a result of Steig’s objections being satisfied, it was published by Steig being replaced by Reviewer D – late in the day, for sure, but a necessary eventuality. I have no doubt that, if Steig had not been replaced, he would be today continuing to obfuscate and delay publication of O’Donnell et al’s paper. And, I have little doubt, smiling and waving in a friendly and supportive way when public-facing.
Apologies if this has been covered already but how prevalant is “double-blind” review in climate journals? Shouldn’t it be the norm given the volatility and politics? I realize this wouldn’t solve everything (or even much) but I find it hard to believe that a paper with a name like Lindzen, Christy etc. wouldn’t get the gears a little harder than others (and vice versa).
Menth — good option to look into. Not really sure, but I’d keep asking around other blogs too, where editors and scientists post.
Keith – Also, Revkin published the entire Derry comment to a new post. Good for Revkin.
At 84, AMac says:
Surely this situation isn’t an outlier, from the editor’s point of view. Author X writes a paper, author Y sees big problems and submits a MS claiming, “X approached this wrong, the issue should be handled as follows”¦” As an editor, I would certainly solicit X’s view of Y’s MS “” who better to point out flaws? But I wouldn’t consider X’s review to be CoI free “” it’s the conflict that makes X’s insights valuable.
The part I made bold is somewhat inaccurate. Ryan and coauthors didn’t write their paper to provide an alternative approach. They weren’t trying to provide a “better” method. The point of their criticism was Steig’s conclusion was an artifact of methodology.
In other words, they were doing a sensitivity analysis, not providing an alternative answer. The distinction doesn’t matter for AMac’s point, but it is important for understanding the conflict.
Derry at Dot Earth:”Finally, Revkin’s point that the Steig vs O’Donnell debate is not unusual in the progress of science and does not have much of anything to say about the majority of the evidence is correct.
Damn straight. But the kabuki blog theater goes on…..
#86 Grypo
“Whether this is true or not, I think it’s important to realize that Steig’s work was the first and “˜first’ works usually changed over time. There was no O10 for the Nature to turn to. I highly doubt Nature would put easily refuted work on their cover if they had a chance to know of anyone who could review it more in depth.”
Not really true. Almost immediately after the article came out bloggers, including O’Donnell, pointed out the statistical flaws. The problem is that “mainstream” climate scientists are not skilled in statistics and apparently do not make much of an effort to check their work with people who are.
JD
This:
> There is the matter of the 88 page critique Reviewer A made of the original O’Donnel paper.
is moot at best:
http://rabett.blogspot.com/2011/02/two-four-88-odonnell-gate.html
My comments on the affair and the response from Delingpole and Liljegren.
JD, Nature didn’t know what the bloggers would say before printing S09. If O10 takes away anything new that S09 suggested, then that would be an advancement of the science. The actual reality of the temperature in Antarctica will likely be shown in boreholes due to the data poor regions that the papers discuss. That was my point to Amac.
“mainstream” climate scientists are not skilled in statistics and apparently do not make much of an effort to check their work with people who are.
This statement needs qualifying. There have been thousands of papers in ‘mainstream’ science that use statistics. In this case we are discussing one. If your statement is true, where is the reviewed literature?
#100 Grypo
My reference to mainstream refers to Mann & Steig & whoever reviewed the article for Nature. Both Mann’s hockey stick article and Steig 09 were published in well-known journals. “Climate scientists” at universities did not pick up the error even though the publications had large circulations. It took bloggers to pick up statistical errors, which they did quickly and easily. That shows to me that the Steig branch of climate inquiry is lacking in statistical skills. It is funny that a statistical failure like Mann was a co-author of the paper and that he didn’t learn anything from his previous failure.
JD
interesting follow up at climate audit…