Nutrition Researcher in p-Trouble

When tortured, the data confessed: "The green ones cause acne!" (Photo by Patrick Fore on Unsplash)

A couple of years ago, I blogged a deliberately bad "study" by "Johannes" Bohannon. His supposed result, that chocolate causes weight loss, was parroted by news media after he headed a study and sifted through the data until he found a meaningless correlation with a high p-value, a statistical term that, in a proper context, can indicate a high degree of confidence that a result is not due to chance.

Fast forward to 2017, and we learn that nutrition researcher Brian Wansink, author of Mindless Eating, a book on food psychology, has turned out to be guilty of "mindless researching," to borrow a phrase from Alex Tabarrok: He has had to retract several papers after being caught using this ... method. Pharma blogger Derek Lowe notes that his academic research came under scrutiny last year after he basically admitted to the tactic on his blog:

But actually, the trouble began with a post on Wansink's own blog. He described the pizza work as initially appearing to be a "failed study" with "null results", but went on to describe how a grad student in his group (at his urging) kept going back over the data until she began finding "solutions that held up". That raises the eyebrow, Spock-style, because you're supposed to design a study to answer some specific question. Rooting around in the data post hoc to see what turns up, although tempting, is a dangerous way to work. That's because if you keep rearranging, testing, breaking down and putting together over and over, you can generally find something that comes out looking as if it were significant. But that doesn't mean it is. (Update: as Alex Tabarrok points out, there is a very germane XKCD for this!) [links in original]

It is hard for me to think of a scientific field that inspires less confidence than nutrition -- Lowe describes many of the reasons for this here -- and that's without hearing about shenanigans like these. Worse, Wansink isn't some fringe academic. Lowe's post ticks off Wansink's credentials early on:

...Brian Wansink at Cornell, where he hold[s] an endowed chair. He's the former head of the Center for Nutrition Policy and Promotion at the USDA, author of both a long list of scientific publications and popular books, and his work is widely quoted when the topic of human behavior around food comes around.

I recommend reading the whole thing, and it links to a couple of other pieces on Wansink that might merit your attention, as well: "But once past the outright misconduct, some of the other activity described is all too familiar, and to see them mixed together in a 'Can you believe this stuff?' article makes for uncomfortable reading."

-- CAV

Updates

Today: Made corrections relating to the emphasis of Wansink's endeavors, which is on the psychological element of nutrition.