0 Logical Fallacies and Scientific Method

Cracked had a very nice article on logical fallacies -- that we all make as a matter of course.  Also some good illustrations and suggestions.  Aside from the fact that it was a humor magazine that had such a nice article on rational thought, I was struck by the fact that each of the points mentioned are ones that the practice of science has addressed.

The 5 natural fallacies mentioned are:
5. We're Not Programmed to Seek "Truth," We're Programmed to "Win"
4. Our Brains Don't Understand Probability
3. We Think Everyone's Out to Get Us 
2. We're Hard-Wired to Have a Double Standard
1. Facts Don't Change Our Minds

Let's take a look at what science method does to combat these:
 
5. We're Not Programmed to Seek "Truth," We're Programmed to "Win"

In science, the 'win' is changed to be the successful seeking for 'truth'.  Out-talking someone, as in a public debate, or out-wording them on an internet argument, or just browbeating them enough that they leave in either, is not a win.  Hence the lack of interest from scientists in 'debate'.  Putting forth an idea that is seen, eventually, to match reality better than what came before is the win.

4. Our Brains Don't Understand Probability

Therefore, we go through the occasionally ugly math to nail down the probabilities of our results.  We just can't trust our intuitions about probability -- our brains don't naturally handle it at all well.  We go back and work through that math, and then after publishing, many people read the news article and say 'everybody knew that already'.  While everybody may have thought it in the first place, we do the work because it's also common for a reader to see the same article and say 'that's absurd, everybody knows it isn't so'. We may not agree that it was obviously true, or true at all, but we can agree on whether 2 * 3 = 6.

3. We Think Everyone's Out to Get Us

In the sense, the article notes, that "If you're smart and savvy, you know not to trust anyone. This is why we can excuse ourselves for using shady or flat-out dishonest tactics to win an argument. We're sure the other guy is doing much, much worse."

So how do you work towards truth if everybody is out to get you?  One part is that when someone is found lying in their work, they're out of the field. Contrast that with, say, business or politics.  Another is to enlist the help of other people who are knowledgeable in the topic and have them read the new work to ensure that there aren't any obvious mistakes or frauds.  Peer review.  Bad papers still make it in to the scientific literature, but it improves the chances that what you're reading is not too badly flawed.  When the peer review process fails, the editors in charge usually take it very seriously.  When's the last time a corporate president resigned because a vice president let a salesman lie about their product?

A different, major, part of the method is that experiments must be repeatable.  Good enough a fake to get past a reviewer is not enough.  Somebody, somewhere, must be able to repeat your experiment and get sufficiently similar results.  For preference, someone should actually do so, but funding agencies don't like paying two or more groups to run the same experiment -- a failing in funding agencies and those who allocate funds for research.  But if the experiment is not even in principle repeatable, if the answer is 'trust me', you're in trouble.

2. We're Hard-Wired to Have a Double Standard
My science example is different than the article's, but the same principle is involved.  It is natural to consider evidence in favor of your position to be better than the evidence against it.  It is so natural that it's also natural to simply ignore the evidence against your position outright, and only look at the evidence that supports it.  Even if you have to make it up, or use for your source someone who did.

To combat this natural double-standard, in science, unlike politics/business/law*/..., you are supposed to present the evidence without regard for whether it supports your position or not.  And if you fail to present evidence that is against it, you're in trouble (#3, #5).

1. Facts Don't Change Our Minds
Being able to do this is what I called the central skill of a scientist.  It is so important because it is so unnatural to us humans.  Quoting some pieces of the original article:

....  Let's go back to the beginning for a moment, and the theory that people figured out how to build arguments as a form of verbal bullying rather than a method of spreading correct information. That means that there are actually two reasons somebody might be arguing with you: because they actually want to get you to think the right thing, and because they're trying to establish dominance over you to lower your status in the tribe (or office or forum) and elevate their own. That means there's a pretty severe cost to being on the wrong side of an issue completely separate from the issue itself. ....
So During Your Next Argument, Remember ...You won't remember this. You're hard-wired to remain entrenched, and the Internet makes it worse because your political beliefs are pasted all over Facebook and wherever else you post your opinions. Backing down means going back on all that. It means letting down your team. Every inch of your psychology will fight it.

Doread the original article in full.

* Law has its own standards on proof and approach to truth.  And it must.  My wife is a lawyer, so we've had some fun talks about the differences and where they came from.  One where it differs most strongly from science is how it handles the natural double standard.  In science, we take the side of making practitioners do the highly un-natural thing of avoiding the double standard.  Law, at least in common law countries like the US (and UK, ...), takes the opposite -- if everyone is predisposed to some level of double standard, and side-taking in their arguments, let's take it out to the extreme -- each side presents the best possible case for its own position, and only that.  Then have a judge or jury assess who made the better case.  The people deciding which is the stronger case are not the ones who make the case in the first place, so (the design hopes) they won't be subject to the double-standard problem.  In science, the same people who would be deciding which case is stronger are the ones making (some of) the cases.

The article helped me understand a conflict I'd encountered.  On one hand, doing science is very natural.  We are all disposed to learning how the universe around us works, starting from birth.  On the other hand, what I see in internet discussions bears strong resemblance to the 5 fallacies discussed above, even when the topic is scientific ("Is CO2 a greenhouse gas?").  Even though trying to find out more about the universe is natural, the methods that evolved over the last few thousand years to help us do so have required us to do it in differently than we reflexively choose.

Detail: Logical Fallacies and Scientific Method

0 Is it science?

One of the things I like to ponder is how to decide whether something is science or not.  An attempt to come up with a clear demarcation criterion is Karl Popper's, which gets more widely distributed as being "If it isn't falsifiable, it isn't science."   I'm not sure what he said himself, but philosophers tend to write books on these topics, rather than short sentences, so I'll guess that some details are lost in this version.

The question arises here because a recent question at the question place (yes, Robert, that's exactly what it's for) mentioned Popper.  I'll give a different response and discussion here.  (Same conclusion*).

For some cases, Popper's falsifiability criterion works well.  Religion is not science.  There is no observation, experiment, or test that will tell someone that their religion is wrong.  No matter what you observe, the religion can accommodate it.  Same thing for mathematics, actually, as it isn't necessarily concerned with observations.  Unfortunately, those (theology and mathematics) are the only two areas which can lay claim to absolute Truth (of a sort -- mathematical truth is only about mathematical things).  Science is left with only approximate truth -- the theory seems to work pretty well, the observations are pretty reliable.  But not absolutely reliable, and not absolutely perfectly.

For others, though, it's more difficult.  In the later 1800s, astronomers observed that the planet Mercury wasn't where it was supposed to be according to Newton's laws.  Its point of closest approach to the sun (perihelion) was moving by 43 seconds of arc per century too much$.  If Popper's criterion were correct, astronomers and physicists should have immediately thrown out Newton's laws and gone looking for something else.  Instead, some patches were suggested -- like a planet 'Vulcan', orbiting even closer to the Sun than Mercury, in just such a way to cause Mercury to behave as observed.  But it was never observed.  Eventually, Einstein proposed his theories of relativity to expand on Newton's laws.  Among other things, they explained why Mercury wasn't where Newton expected it to be.

For climatology, Popper is not so much relevant, or at least doesn't pose very much difficulty.

Popper's criterion is mostly concerned with theories, not observations.  And climatology has few theories of its own.  It borrows theories from other areas of science -- the laws of conservation of mass, energy (first law of thermodynamics), and momentum (Newton's laws), the second law of thermodynamics, Planck's law for blackbody radiation, quantum mechanics (to get emission and absorption of radiation by gases), and a few more subtle matters.  Climatology also makes some use of observations -- where are the absorption lines of CO2 and how strong are they, what is the saturation vapor pressure of H2O, what is the freezing point of water, and so on.

To the extent that people complain about climatology, and climate models, the theory (theories) that have to be wrong are from somewhere else (usually physics).  The climate models are just trying to carry out the laws and theories from physics.

Popper's criterion, to come back to it, doesn't have much to say about observations.  I am 6'1" (185 cm) tall.  At least I claim that this is true.  Suppose you come to my house and measure my height and see that I'm, instead, 184 cm tall.  Maybe you've 'falsified' my claim.  But what has changed in the world of science?  Nothing.  No portion of science relied on me being 185 cm tall rather than 184 cm tall.  Certainly far less depended on that than depended on Mercury's orbit, and that didn't, on its own, cause the downfall of Newton's mechanics and gravity.

But the law(?), theory of conservation of energy is something to falsify.  It _could_ be falsified.  If you observed the amount of energy in a system at one time, and then some time later saw that it had less (or more), you'd be on your way to a Nobel prize.  Or at least a new revolution in science.  One of the major importances of Einstein's E = mc^2 was that it showed there was a new place to find energy in the universe.  Odds are good, given the 100+ years that people have been at it, that you've just made an error in your observation process.  Still ... maybe you've got something.

Rather than drive on to a firmer conclusion, I'll stop here and invite discussion.  What theory or theories do you think climatology has that couldn't be falsified?  Which ones do you think have already been falsified?  If you think climatology is making claims which could not, even in principle, be falsified, what are they?  Is Popper's criterion even a good one to use?  And so on. 


* There's an old story that goes:   A student who was not doing very well in a class discovered that their professor always gave the same test every year.  Even better, it was multiple choice.  So they tracked down an old, corrected, copy of the test and memorized the answers.  Come the day of the final, they checked off all the previously-memorized answers.  Then was extremely surprised to have gotten an F (failing grade).  After some internal debate, the student went to the professor and said what they had done.  The professor answered "Yes, I give the same questions every year.  But I change which answers are correct!"

$ 43 seconds of arc is a seriously small number.  A circle has 360 degrees.  Each degree has 60 minutes of arc.  Each minute of arc has 60 seconds of arc.  The difference was 43 / (360*60*60) of a circle -- 0.00003318 of a circle (about 33 parts per million, far less than the fraction of the atmosphere that is CO2, and small enough that the thickness of the line when you draw a circle is much larger than this).

Detail: Is it science?

0 If I were in charge?

Carrot eater asked me to consider what I would do if I were in charge of climate research.  I assume that he wasn't going for answers like 'find a different job promptly', which does make the question a little more theoretical.  Although I do have the copy of Nature that prompted his question, I've not read that article.  So these are my own thoughts.

My first thought is the least creative -- pretty much what is already being done in pretty much the proportions it is already being done.  No doubt that I would like to make some adjustments, say more for ice-related work.  But the main lines have gotten to be the main lines because they consistently show up as areas that deliver improvement to our understanding (satellites, paleoclimate) or they consistently show up as areas hampering our understanding (clouds).  Some areas probably get more funding than ideal, or less than ideal, because humans are involved and a particularly good, or bad, field leader can have effects beyond just writing good papers and proposals.

The two that I like for creative work should start as minor niches.  If my intuition is right, they'll grow markedly, at least for a time.  Because if my intuition is right, there's a lot to be learned from here that would be useful.  But it is pretty much just my intuition, so the starting investment shouldn't be huge.

The less exciting already has some work being done, at least in related fields.  Namely, 'no approximations' modeling.  We do know the equations that describe how fluids move, for instance.  We can write programs that carry out those equations accurately.  But once you're examining a volume of fluid larger than a moderately large fish tank (call it 50 gallons, 200 liters), you have to make approximations.  Computers can't deal with the full dynamics for a larger volume than that.  Nevertheless, in the 1950s and 1960s especially, quite a lot was learned about the general circulation of the atmosphere by doing 'dishpan' experiments.  Dishpans can be set back up, and the computers given accurate representations of them.  And then we can see how close the models come to the observations. 

More about the dishpans in a later post; they were a very clever way of approaching the atmosphere.  But here's a modern version's photo:
  and see also the original press release about the memorial lab, from which I got that picture.  I was among the last students Fultz fired up his working lab for.

The more exciting, to me, notion turns on an observation that I find very striking, and very few others in the field find at all interesting.  Makes it a high risk idea -- those other people are awfully smart, good chance they're seeing a flaw that I'm not.  I'll have a little explaining to do, but the short form of the observation is: for something like the global climate surface temperature field, you only need something like 12 numbers.

If you look at our climate models, they have quite a lot more than 12 numbers to them.  They've got at least 6 for every grid cell, and several hundred thousand to several million grid cells.  Tons of numbers, and how many increases every time we run a climate model at better (meaning smaller grid cells) resolution.

Yet 12 or so is, in some sense, 'enough' for temperature.  (Another 12 for pressure, maybe a bit more for each of wind and humidity.)  The sense starts here: if you look at temperatures around the globe, most of what you see is climate -- that the poles are cold and the tropics are hot.  So we subtract out climate in looking at the global temperature field.

After doing that subtraction, you can see that areas that are warmer than usual tend to be fairly large, millions of square km.  Ditto the areas that are cooler than usual.  Continue with that sort of analysis and you also notice that the warm areas are usually balanced off by cold areas elsewhere.  Keep doing it and you see that the same areas tend to be counterbalancing each other.  These are areas that you can look at and know even thousands of km away whether it is warmer or cooler than usual.  The El Nino-Southern Oscillation is the best-known of these patterns.

Once you get rigorous about doing this, which I will be for a later post, you find that if you pick the right areas, once you've chosen about 12, you can put together the entire globe to fairly good accuracy.  This is the root of methods to reconstruct paleoclimate from information sampled from only a modest number of locations around the world.

But I look at it and think, and wonder ... if you can describe something with only 12 numbers, global surface temperatures in this case, then there should be some way to model it with only that many variables.  If only we were clever enough?

So one thing I'd like to see some funding for is these sorts (I'm sure I'm not the only one with such ideas) of basic research ideas.  They're ideas that will probably take a few years to develop, and most will never pan out.  But they're also the ideas that will revolutionize the field if they do pan out.  Current support for science has gotten very short term in its view.  If you can't be sure to publish a paper or three per year from a grant, you are very unlikely to get it.  Fields therefore get very 'play it safe' in what they attempt to do.

Detail: If I were in charge?

0 Theory of Climate -- Philosophy

Is there a theory of climate, and if so, what is it?  That turns out to be a harder question than you might think.  It's my slight rephrase of a question asked in this month's question place.  The difficulty lies in the fact that 'theory' has several different meanings.

We can dismiss the most common daily life sort of usage -- a theory is something that is false.  It's used in comments like 'That may work in theory, but it doesn't work in practice.'  If we're talking about what happens in practice, we're talking about what happens in the real world.  Scientific theories have to apply to the real world.  If your idea makes false predictions about the real world, then the idea is (at least partly) false.  And, if your idea consistently makes false statements about the world, then it is not a theory, or even a hypothesis.  If there is a scientific theory of climate, it must be making true statements.

We can also dismiss the next most common daily life usage -- a theory is a WAG (wild guess).  A common sort of theory of this type is where I, for instance, theorize that since the last time I took my car to the shop, it got good service, that the next time I go, it will as well.  There's really very little data behind that thought, and very little analysis.  But it seems like a reasonable sort of statement.  Or the 'lucky socks' theory sports fans or athletes might have.  Their team won the last time they wore a particular pair of socks, so they theorize that the team will win again (or at least have a better chance of winning) if they wear the same socks for today's game.  Again, it seems reasonable to the person making the statement, but there's little data behind it and little analysis.

Now to consider the more difficult waters, where I hope that the two philosophers I know sometimes read will comment with appropriate corrections and elaborations.

One part of math/engineering/science usage is that, in contrast to common usages, a theory is a very good thing indeed.  Actually the best you can get.  Some other terms you'll hear are conjecture, hypothesis, and law.  The 'good service' or 'lucky socks' theories, as we'd term it in common conversation, are really more in the nature of a conjecture.  We've got very little data, and very weak chains of reasoning.  But, hey, it looks ok.  After you collect more data and get stronger lines of reasoning, you have a hypothesis.  It's also better if the hypothesis covers more situations than the conjecture.  Get to the point of covering many situations, with very strong evidence and very strong lines of reasoning, and you finally have a theory.

Law is a different kettle of fish entirely.  In terms of how we do science any more, it's also an anachronistic term -- something that used to be used, but now is uncommon.  Laws, in my areas at least, are simple mathematical relations between a couple of things.  Newton's Law of Gravitation, for instance, says that the gravitational force between two bodies is proportional to the product of their masses divided by the square of their distance.  Simple relation and it's even reasonably correct.  Hard for you to test for yourself, so I'll mention another one.  Hooke's law of elasticity.  This says: take a spring (or rubber band, or piece of elastic, or bungee cord, ...) and hang it up.  Measure where the end is.  Now put a small known weight at the end and measure how much longer the spring gets.  Add a second weight, equal to the first.  You'll see (says the law)  the spring lengthen by exactly the same amount as before.  This is reasonably correct as well.  But as you keep increasing the weight, you'll eventually see the distance the spring lengthens start to change, and eventually it changes quite dramatically -- if you put enough weight on the spring, it will break.  Laws in this sense are convenient expressions of relations.  You can use them, within limits.  But there definitely are limits and, often, even the proposers knew about the limits (I think Hooke did).

One place you can see the change in how scientists think/thought about Law versus Theory is that Newton's Law of Gravitation (1600s) was replaced by Einstein's Theory of General Relativity (1910s).

Within the realm of scientific theories, though, there are two different sorts.  I'll call them the elegant and the complex.  We mostly think about the elegant theories when we think about scientific theories.  The theory of evolution, for instance, is an elegant theory.  Its elegance lies in the fact that it covers an enormous range of observations, can be used to make a large number of predictions, and, yet, can be stated in a page.

I don't believe there is an elegant theory of climate.  It is of the complex theory sort.  I have in my library, for instance, a book on Number Theory, and one on Theory of Flight.  In neither will you find a one page description of the Theory of Numbers, or the Theory of Flight.  The sense of theory involved is 'a body of principles and understandings that apply to a given subject'.  It's an elegant theory when the number of principles or understandings is small.  And, for something like flight, or numbers, and on for many areas, you have quite a few principles to work with.

In climate, some of our principles, or statements of theory, would include:

• Conservation of mass applies to the climate system
• Conservation of momentum applies 
• The laws of thermodynamics apply
• The general theory of relativity applies
• Quantum theory applies
• Tectonic theory 

I'll take up illustrating how these apply to Theory of Climate tomorrow. In the mean time, I'll invite discussion of the philosophy and additional examples of theories or laws that are part of a Theory of Climate.

Detail: Theory of Climate -- Philosophy