A Pound of Gold or a Pound of Hay?

Wednesday, June 13, 2012

Jonah Lehrer of The New Yorker claims that psychological research shows that "[t]he vast majority of people respond quickly and confidently" -- with the same wrong answer -- to the following simple problem:

A bat and ball cost a dollar and ten cents. The bat costs a dollar more than the ball. How much does the ball cost?
Upon reading the above sentences, I immediately saw a simple algebra problem. In mathematical terms, the first two sentences could be expressed as:
a + b = 1.10 (1)

a - b = 1.00 (2)
In the above equations, a represents the bat and b the ball. I didn't bother with the equations, and didn't need to, anyway, to see that the "popular" answer of ten cents (which the problem statement might suggest to someone in a hurry) was wrong. Such a price would lead to a contradiction between (1) the total price of the two items and (2) the stated difference in the price between the two. A price of one dime for a ball would mean that the bat would have to be worth (1) $1.00 and (2) $1.10 at the same time.

The article goes on to note the following:
And here's the upsetting punch line: intelligence seems to make things worse. The scientists gave the students four measures of "cognitive sophistication." As they report in the paper, all four of the measures showed positive correlations, "indicating that more cognitively sophisticated participants showed larger bias blind spots." This trend held for many of the specific biases, indicating that smarter people (at least as measured by S.A.T. scores) and those more likely to engage in deliberation were slightly more vulnerable to common mental mistakes. Education also isn't a savior; as Kahneman and Shane Frederick first noted many years ago, more than fifty per cent of students at Harvard, Princeton, and M.I.T. gave the incorrect answer to the bat-and-ball question.
Over half of these students failed to spot a simple contradiction! Assuming the people surveyed were not told to hurry to give an answer, this suggests that an incredible number of people out there do not, as a matter of course, integrate one item of knowledge with another. This is astounding to me on one level, although the fact that people who fail to do this are also more prone to cognitive bias is not.

An explanation I would have liked to see considered is what kind of thinking methods people have been taught or exposed to over the course of their educations. "Progressive" education, informed by the anti-integrative philosophy of Pragmatism, has profoundly affected much of American education for decades. I suspect that if such an explanation were considered, the fact that education doesn't generally save people from such slips would seem much less surprising. Furthermore, the fact that intelligent people are more prone to cognitive bias would also make sense, given the fact that most spend more time than average with the comprachicos.

-- CAV

12 comments:

Steve D. said...

Of course the entire study as well is predicated on the fact that those people took the question seriously. The use of multiples of ten in the problem is designed to trick them into thinking they don’t have to think carefully. If you gave them a slightly more difficult question, which would force them to take more time, I would bet that almost everyone would get the correct answer. In other words the reason people failed to get this correct is laziness or impetuousness and has nothing to do with their thinking skills.
I would also bet that if you reversed the two statements in the problem, a larger percentage of people would get the correct answer
I’m going to ask my son this question and see what happens.

Realist Theorist said...

This example is quoted in the book Thinking: Fast and Slow, by Daniel Kahneman.

The explanation given is that we sometimes spend the time to consciously work out some solution (thinking slow), but we often just try to come up with the answer with only the briefest of "thought" (thinking slow). If this problem were to be phrased with different numbers (e.g. $13.21 etc.) people will stop and take the time to work it out; otherwise, the temptation is to take the answer thrown up by first impression.

Kahneman says that we automate all sorts of thinking and action (think driving) and that this is a good thing. We could not go through life without this. As part of our automated processes, we also have some automated way to alert us to situations where we should go off auto-pilot. However, by the nature of the automated beast, we are going to have situations where we react using the automated guesses and heuristics when we ought to be giving the situation more conscious thought.

I recommend the book, along with "Influence", by Cialdini.

Anonymous said...

This reminds me of the reader response many years ago when Marilyn vos Savant gave her answer to the Monty Hall (or "Game Show") Problem.

Her answer and the responses are still posted on her website. What's fun is that she re-answered the problem two or three more times, and each time was still getting furious responses. Many came from mathematicians.

http://marilynvossavant.com/game-show-problem/

Gus Van Horn said...

Steve,

Let me know what happens with that.

RT,

Thanks for fleshing out Kahneman's thesis. Your point about things that alert us when we should not use heuristics is very interesting and will tempt me to read the book when my hopper (and the floor!) is less full.

Anon,

Rushing through four comments this evening... This looks amusing, but it'll have to wait. Thanks for the link.

Gus

Steve D. said...

Most of the people I asked suspected immediately that there was more to the question than met the eye. Everyone (ten out of ten) got the correct answer within about ten seconds or so, even my son who is eleven years old. A couple people admitted that the incorrect answer 'nothing' briefly flashed through their minds at the beginning of their thought process.
I also must admit that for a few seconds when I first read what you wrote, I thought the ball had to be free as well. (Of course there is no such thing as a free ball or free lunch, right?). I think what happened was that I continued reading your next paragraph and so did not give the problem the attention it deserved. However when I wrote out the question and presented it directly to other people, the fact that I was doing it made them suspicious and so they thought about it longer before returning their answer.

Gus Van Horn said...

"Most of the people I asked suspected immediately that there was more to the question than met the eye. "

I'm guessing -- based partly on memory from the New Yorker piece -- that the researchers got around this by slipping the questions in with others.

Steve D. said...

‘This reminds me of the reader response many years ago when Marilyn vos Savant gave her answer to the Monty Hall (or "Game Show") Problem.’
This is a classic case of adding information into the system thus changing the parameters. Just like the ball and bat problem, a person should immediately upon hearing the question expect there is more to it than meets the eye. (What other reason would there be for asking the question?) Similarly, at first the incorrect answer flashed (angrily) through my mind, but it took only a couple minutes to figure this one out (ten times longer than the ball and bat). Had I not that context and instead met the problem cleanly without any reason to expect trouble though, I probably would not have figured It out.
So, I have a little bit of trouble believing that all of those angry emails are real, especially after her first explanation (which makes understanding it a slam dunk, straight from a stats textbook). I distinctly remember being given problems much like this one by my statistics professor (or perhaps it was my calculus professor, I can’t exactly remember anymore)
Another interesting point is that nowhere in the responses to her answer did I see any explanation about why her explanation was wrong. I would have been very interested in hearing the response from the people who agreed with her.

Gus Van Horn said...

"Just like the ball and bat problem, a person should immediately upon hearing the question expect there is more to it than meets the eye."

This is why the high miss rate for the bat and ball problem puzzled me -- unless being posed to someone in a hurry -- and then you get no way to record whether someone had a nagging feeling that there was (e.g., manifested as a second thought or a desire to check the answer.

Realist Theorist said...

Gus,

You're right that the question is posed to people "in a hurry". The fundamental thing this experiment is trying to illustrate is that our lightning-like, automated, pattern-matching thoughts throw up the wrong answer.

There's no doubt that if the same question were asked on the SAT a higher percentage of people would get it right.

In addition, the better a student's grasp of math, the more automated his "algebraic" approach to the question, and the less chance he will be tempted by the easy-answer. This was shown by other, similar experiments.

There is a "cost" to thinking, and the context of the question, the cost of giving the wrong answer, and the student's own efficacy at the process by which he can get the right answer, all play a role in whether he goes with the first thing that pops into his mind, or whether he pauses and thinks about the answer.

I should add that Kahneman's book is mixed. It could easily have been half its size, and there will be times where you will disagree vehemently with his interpretations. Nevertheless, it is well-worth the read, because it has a wealth of such examples... and enough material to let you draw your own interpretation.

Gus Van Horn said...

Thanks for the further clarification, RT.

Steve D. said...

‘I'm guessing -- based partly on memory from the New Yorker piece -- that the researchers got around this by slipping the questions in with others.’

But this only strengthens my conviction that it’s a simple matter of apathy or carelessness. (Or perhaps just being in a rush which was also suggested) “Well can we please get this boring math exam over with and back to my beer.”, or shopping or whatever it was that they were doing. Slipping in the question with others solves one problem but creates another.

I also hope the researchers had enough sense to have someone who knew nothing about the experiment hand out the questions. (Double or triple blind is essential)

In any case, merely the fact that it’s a survey rather than real life, will I’m sure change how many people approach it. As you can tell, I’m not a great fan of using surveys or sets of questions to learn about human behavior. It may tell you a great deal about how people do on surveys but if their survival (or retirement fund) was at stake, I’m predicting they might take a little more care.

Gus Van Horn said...

As you point out, the relationship between values and cognition is essential to consider -- and I've had similar problems with other approaches.