My wife, a writer, and I have a fair number of discussions about the creative process. For all that science and literature are 'supposed' to be opposites, dislike each other, and so on, we find that there's a tremendous similarity. Regardless of which you are pursuing, an important question to ask yourself is "Do I have something good?" Are your characters interesting? Have you covered every loophole that could take down your hypothesis? More generally, "Is it art?", "Is it science?".
One of my feelings is that science is about trying to understand the world around us. In particular, since I'm a physical scientist, to understand the natural, physical world around us. One sort of question we could ask about the natural world is "What have global mean surface temperatures been like for the last 100 years?" As you all know, there are controversies about that.
What strikes me, though, is that most of the controversy I encounter in the media, in the blogosphere, or elsewhere, is not about the science.
The most popular vein of controversy is efforts of people to 'audit' the work of people who were trying to answer the scientific question. And that mostly baffles me. As a scientist, it's entirely baffling. As a citizen, I see reasons, some good, some not.
The scientist in me is baffled because "let me have all your data sets and all your programs so that I can see if I get your results" does nothing at all to answer a question about the global mean surface temperatures. No matter what is done, whether the exact same answers are found or not, we know nothing more about what global mean temperatures have been. That makes for a bad scientific question.
In a good scientific question, you learn something regardless of what the result is. The science on global mean temperatures is not in 'auditing', but in doing more science. Ask questions like "The prior work did their spatial gridding in a very simple way. Do I still get an answer like theirs if I use better gridding methods?" "The prior work trusted some stations I think are very untrustworthy (insert list of objective reasons here). Do I still get an answer like theirs if I exclude those stations?" "We now have much better data from this new source. Does it show the same trends as the old source?"
Sometimes your answer is a boring 'yes'. Ok, your gridding method is better, and you didn't use some of the bad old data sets, and you even made the jump to include some spiffy new data source, but still you got the same old answer. Reassuring that the original folks were pretty much correct. But boring.
And sometimes you get the exciting 'no' -- you have a significantly different answer. This played out in the late 1990s and early 2000s between the surface station record and the then-new satellite sounding temperatures. In 2000, the discrepancy was still present and uncomfortably large (outside the error bars of either the surface record and the satellite record). That occasioned a National Academies panel discussion and summary http://www.nap.edu/catalog.php?record_id=9755 of the disagreement. At the time, there was no hard conclusion about cause, basically just outlining the understandings. It developed later that the reason for the discrepancy was largely errors in how the satellite data were being analyzed (see for example Mears and Wentz 2005).
My virtual bet was that this would turn out to be the case. Not because the surface record is great, because it isn't. Nor that the satellite guys were doing wrong things (not knowingly), but because they had only been at it for a few years. New observing systems, or old systems being used in new ways, take time to shake out bugs in how they're used. Surface records had been used for over 150 years by 2000, the satellite had only maybe a decade. So things were learned and we now have a new observing system from satellite that we can use to add to our understanding about the atmosphere. Science happened. Just not the pretty kind we normally talk about in school.
But is it science? I find this question simplifies my reading greatly. So many articles are not about the science. If I'm interested in learning about what we know or how well, it has to be an article about the science. Articles that are, instead, about how person A doesn't like person B, how the feeling is mutual and all that sort of thing ... even if A or B are scientists, that article doesn't have anything to show me about the science.
It's also a good question to ask yourself. I had a notion in graduate school regarding snow formation that I started describing to a friend (who was studying snow). It wasn't too long before he asked me where the science was. Plenty of work, but where was the science? I never really did find it for that idea. Somewhere near where I was thinking, I'm sure there's a good research project. But it's outside where I've thought.
