0 Peer review and Wagner Resignation over Spencer and Braswell

"It's peer review, not God review"    My wife's comment about peer review seems particularly apt for the current tempest regarding the resignation of Wolfgang Wagner from his post as editor in chief of the journal Remote Sensing.  It regards a paper I mentioned in July, and related to the one that prompted Barry Bickmore to suggest "Just Put the Model Down, Roy". 

I won't be taking the usual line of consideration here (surprise!).  Rather, let's go back to talking about peer review.  As my wife said, it is not God review -- reviewers and editors are human, and therefore make mistakes.  At the end of peer review, therefore, you don't have gospel, you have something that has a fairly good chance of being worth your time to read.  To rephrase Wagner, papers that pass peer review should at least not contain fundamental errors of method or false claims.  And it now seems likely to him that this paper (Spencer and Braswell) may well not pass that standard.

Richard Feynman's comment about fooling yourself is commonly quoted:
We've learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature's phenomena will agree or they'll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven't tried to be very careful in this kind of work. And it's this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in cargo cult science. "Cargo Cult Science", adapted from a commencement address given at Caltech (1974)

This underlies some parts, I think, of the failure in Spencer and Braswell's work, and the subsequent failure in the review and editorial process.  Namely, it is assumed by reviewers and editors that authors have already done some work doubting themselves and checking to see how it is they might have fooled themselves -- and to take action against such possibilities.  Further, the review process is based on the presumption that your purpose in publishing is to advance our understanding of science.

Wagner mentions one of those non-scientific aspects to the paper -- the public exaggeration of the paper's conclusions by the author himself.  What the university press releases and certain media outlets do in exaggerating is pretty much outside the author's control.  But Spencer's own exaggerations indicate (my thought) that his purpose was to make those grand claims, rather than to make an incremental improvement to our knowledge of how the universe works.  As Wagner mentions, no single paper looking at a single data source, comparing with a single model, is capable of refuting all science on global warming.

A different non-scientific point, endemic to any of Spencer's work, is that he doesn't fundamentally view himself as engaging in science and an effort to understand the universe; he says "I view my job a little like a legislator, supported by the taxpayer, to protect the interests of the taxpayer and to minimize the role of government."  Note that -- his job.  There are scientists who pursue political goals, whether to minimize government, or prevent what they think are bad political decisions -- but it's on their own time, as private citizens -- not as their job.  The particularly active ones I know have always had a clear distinction between their job and their activities as a private citizen.

With Spencer, though, he views is job as being to achieve a political end.  As such, science takes a back seat.  This paper of his, then, is less a matter of trying to advance our understanding of the world, and more (as he views it as his job) a matter of trying to achieve a political result.  Peer review was not developed for such things, and does not manage them well.  When science is not the primary purpose of a paper, that presumption reviewers and editors make that the author has done his due effort at avoiding fooling himself is false.

Some comments have been made hither and thither about how the criticisms, like Bickmore's, of Spencer's work are not in the peer-reviewed literature.  They then conclude that this means that Spencer is right and Bickmore (and others) are wrong, or the criticisms can, at least be ignored.  If the authors have done their due efforts to not fool themselves, this may well be true.  In such efforts the fundamental errors Wagner mentions would not be present.

Instead, as was the case for McLean et al., some errors were so elementary that they could be analyzed and discussed at blog-level.  Granted I may often leave some of the middle-school kids behind, but Bickmore's analysis, for instance, should have been manageable by anybody with an undergraduate major in a math/science/engineering field.

There's a flip side to that.  You seldom can publish things at that level in scientific journals.  They're 'too obvious', or 'uninteresting', and the like.  Consequently, if the errors in a paper are elementary enough, there will be no peer-reviewed literature to cite regarding the error.  Authors are supposed to have done that checking themselves, and if they fail to, the reviewers are supposed to mention them.  If both fail, well, now we do have the possibility of writing blog comments on the failings of a paper. 

For comments and discussion here, I will encourage you to focus on the peer review process itself.  I give links below for the science in the paper, and others for the story itself if you're more interested in those things.


Some links on the science:
http://www.realclimate.org/index.php/archives/2011/07/misdiagnosis-of-surface-temperature-feedback/
http://profmandia.wordpress.com/2011/08/03/spencer-braswell-2011-proof-that-global-warming-is-exaggerated-or-just-bad-science/
http://bbickmore.wordpress.com/2011/07/26/just-put-the-model-down-roy/

Some additional links on the story:
http://initforthegold.blogspot.com/2011/09/editor-apologizes-for-spencer-paper.html
http://www.skepticalscience.com/news.php?n=982
http://scienceblogs.com/deltoid/2011/09/editor_of_remote_sensing_agree.php?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+scienceblogs%2Fdeltoid+%28Deltoid%29
http://www.forbes.com/sites/petergleick/2011/09/02/paper-disputing-basic-science-of-climate-change-is-fundamentally-flawed-editor-resigns-apologizes/
http://retractionwatch.wordpress.com/2011/09/02/editor-of-remote-sensing-resigns-over-controversial-climate-paper-co-author-stands-by-it/
http://scienceblogs.com/stoat/2011/09/holy_editor_resignation_batman.php
http://rabett.blogspot.com/2011/09/honor-and-respect.html
http://blog.chron.com/climateabyss/2011/09/spencer-braswell-and-the-review-process/
http://bigcitylib.blogspot.com/2011/09/editor-steps-down-for-publishing.html
http://scienceblogs.com/gregladen/2011/09/cloudgate_denialism_gets_dirty.php

Spencer's blog and his comments on the news