We Don't Always Need More Information

Wednesday, April 03, 2019

Statistician John Cook describes a phenomenon he often encounters in his consulting practice. He calls it missing information anxiety:

He couldn't see the forest -- or the way out -- for the trees. (Image by DarkWorkX, via Pixabay, license.)
I often tell clients they don't need information that they think they need. That news may come as a relief, or it may cause anxiety. I may tell a client, for instance, that missing data cannot change a conclusion, so it's not worth waiting for. Whether that brings relief or anxiety depends on whether they believe me.

There's a physics demonstration where you have a heavy ball on a long cable. You pull back the ball like a pendulum and let it touch your chin. Then let the ball go and stand still. If you're convinced of the physical laws governing the motion of the ball, you can stand there without flinching. You know that just as it left your chin with zero velocity, it will return with zero velocity... [bold added]
This is a valuable point. Mathematics is highly abstract, and the contention that no more data is needed might sound ridiculous to a non-mathematician. (This problem is not restricted to mathematics, but it is surely common among similarly abstract disciplines.) Cook speaks of putting "your own own face on the line before asking them to do the same," which is a good metaphor for tying one's abstractions to reality. Or, as one so often hears in communication advice: "Show, don't tell."

-- CAV


Dinwar said...

This reminds me of an experimental design called "Strong Inference". To summarize, you establish mutually exclusive working hypotheses, and develop an experiment that will rule out all but one. This sort of experimental approach is useful when data are limited, such as in paleontology or astronomy (where re-running the experiment may be impossible), because a single datum can be sufficient to establish which hypothesis is valid. The devil, of course, is in the details--you need to properly frame the question!

The other thing this reminds me of is the rampant misuse of the mathematical concept of proof. I follow Gould's view: if you have enough evidence that withholding acceptance of a concept is perverse, you can call it proven, sensu vulgate. This eliminates the line of argument "You can't KNOW that, not absolutely!" You often don't need to; you need to have sufficient certainty to act on the concept. How much depends on what the thing is, of course--we want more certainty for a medical diagnosis than we do for determining when a hamburger is cooked.

Gus Van Horn said...


Thanks for mentioning strong inference. Either I hadn't heard of it or had forgotten about the name. (The approach makes lots of sense, though, so I can't say for certain either way.)