What was at hand was, on one hand (it does help to have many hands if you're in science), a fairly straightforward piece of engineering. On the other hand, a bit of science. Remember that I think both engineering and science are good things, if different. Engineering is mainly aimed at 'apply what is known to achieve benefit for someone', while science is aimed at 'try to understand more about the universe'.
Back in 1993, I was at the National Meteorological Center (NMC), the part of the National Weather Service (in US -- NOAA) that develops the new weather forecast models or tries to make the old ones better. My area was sea ice. Now, one thing we sea ice, polar oceanography, polar meteorology people were entirely confident about was that sea ice mattered, a lot. For, well, everything, or at least enough. If we didn't think it mattered, we'd hardly be spending our time studying it. People outside our little community, including folks working on numerical weather prediction, didn't think sea ice mattered for much. And, if it did matter, surely it was only something that mattered for long time modeling -- climate scale forecasting. Surely the ice was already well enough represented to be good enough for weather prediction purposes.
Partisan as I was, and am, in favor of sea ice, I must confess that there were (and are) good reasons to believe that for short range forecasting, you didn't need very accurate representation of sea ice. It doesn't cover much of the surface area of the earth. And, while it might be very reflective, at the times that there is the most ice that is most reflective, there isn't much sun for the ice to reflect. I could have simply sat back in a wrangle with the weather folks, endlessly asserting that sea ice was important, and how much energy sea ice reflected was still important, and weather is chaotic so it had to matter, vs. endless repetitions of their counter-arguments. Perhaps you've seen that sort of thing happen a time or two on a blog or two.
Instead, time to do some science. Run the experiment and see what happens. This has the downsides that it requires my time, and I have to run the risk of the experiment showing that I was wrong -- that modest changes to how much of the sun's energy sea ice reflects really did not affect weather.
But that, seriously, is what makes it a good experiment to run. I wasn't certain, nor was anyone else, exactly how it would turn out. If I turned out to be wrong, then there's a contribution to our understanding of the universe -- indeed weather for a few days really doesn't care a lot about exactly how reflective sea ice is. It was previously assumed and expected that this was the case. But here, finally, would be evidence that the common assumption was correct. If I turned out to be right, and those small changes did matter to weather forecasts, then the contribution is that not only climate (which everyone agreed was sensitive to such things) but weather as well cared. Me being right or wrong is not where the science sat (much as I would prefer to be right, of course). Whether sea ice reflectivity (albedo) mattered for even short time scales is where the science was.
So I ran the experiments. That was a plural because you need more than a single forecast to decide whether a change is for the better. Weather is chaotic, which means, among other things, that things just happen. You could be a little better in the forecast for no reason of skill, but just because the chaos ('the butterflies' we often call it) kicked things over better. I ran experiments -- 5 day forecasts starting on different days. I chose 40 days, from March through June (10 days per month). The 10 days per month, for months spanning the seasonal transition from spring to summer (northern hemisphere) or fall to winter (southern hemisphere) were a large enough collection that the butterflies couldn't swamp the results, and we'd be able to see how much sun-dependence there was.
To represent the reflectivity of ice, I used work done elsewhere (Ross and Walsh, 1987 -- I was working in 1993) as my estimated improved version. That was to replace a representation that dated back to 1964. As usual, we hope that the additional observations that 20 more years of science had would make for a better representation. But, you still have to test it.
As it turned out, the new reflectivity representation really did improve the weather forecasts. See A sea ice albedo experiment with the NMC Medium Range Forecast Model for the gory details.
So the short term good news was both that my prediction was shown to be right, and, not quite as short term, I got to publish an article making an addition to what we understood about how weather works. As it worked out, it was longer before the last leg of good news happened. But it did -- my suggestion for change was incorporated in the NMC's official medium range forecast model.
There was a post script to the story. After the change went in to the operational weather model, I heard from marine forecasters. They had noticed that storms heading up towards the Arctic, particularly those heading past Iceland, were being forecast better by the model. This made their jobs easier. Not that they ever entirely trust the model, but it makes their job easier, and they can do it better, if they are working with the last 30 km of correction, rather than something like 300 km range. (I make up the numbers, the important reality being that the better the model is in providing the first guess, the more accurate the forecaster refinements are.) I looked back to my experimental runs and realized that most of the skill improvement was from the occasional (one or two out of 10) forecast that was very much better at predicting a storm's path.
Never have written that up, though it's been something I've relayed to other people in the 15 years since then.
In the note here, I've focused on the engineering side. The science details are in the paper. But there are a few good illustrations about how science works:
- There was no drama
- A modest improvement was made
- Interesting things were learned afterwards
I'll note this article is the outgrowth of a comment, and my reply, over at A Few Things Ill-Considered (neither having much to do with the main post, the initial comment is at #6, my reply at #8).
