A must-read article at io9:
I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here’s How.
Other than those fibs, the study was 100 percent authentic. My colleagues and I recruited actual human subjects in Germany. We ran an actual clinical trial, with subjects randomly assigned to different diet regimes. And the statistically significant benefits of chocolate that we reported are based on the actual data. It was, in fact, a fairly typical study for the field of diet research. Which is to say: It was terrible science. The results are meaningless, and the health claims that the media blasted out to millions of people around the world are utterly unfounded.
Here’s how we did it.
The story of how they got their study published and turned into blaring, inaccurate, weight-loss-promoting headlines across the world make for fascinating and worrying reading.
But it’s not just about the reporting. There’s a hugely important message about weight-loss “science”:
I know what you’re thinking. The study did show accelerated weight loss in the chocolate group—shouldn’t we trust it? Isn’t that how science works?
Here’s a dirty little science secret: If you measure a large number of things about a small number of people, you are almost guaranteed to get a “statistically significant” result. Our study included 18 different measurements—weight, cholesterol, sodium, blood protein levels, sleep quality, well-being, etc.—from 15 people. (One subject was dropped.) That study design is a recipe for false positives.
…
So why should you care? People who are desperate for reliable information face a bewildering array of diet guidance—salt is bad, salt is good, protein is good, protein is bad, fat is bad, fat is good—that changes like the weather. But science will figure it out, right? Now that we’re calling obesity an epidemic, funding will flow to the best scientists and all of this noise will die down, leaving us with clear answers to the causes and treatments.
Or maybe not. Even the well-funded, serious research into weight-loss science is confusing and inconclusive, laments Peter Attia, a surgeon who cofounded a nonprofit called the Nutrition Science Initiative. For example, the Women’s Health Initiative—one of the largest of its kind—yielded few clear insights about diet and health. “The results were just confusing,” says Attia. “They spent $1 billion and couldn’t even prove that a low-fat diet is better or worse.”
The ironic thing about Attia’s “lament” is that the lack of clear evidence about diet and health didn’t stop him cofounding an organisation whose website cries out for proper research and funding – but still assumes that the “dramatic rise in obesity” in the US is a “crisis” and not a blip caused by changing the definition of “obesity” in the 90s.
The authors of this terrible study did it deliberately to make a point about bad science and bad science reporting. There are plenty of others out there doing it for a far worse reason: the ridiculous amount of money there is to be made from promoting health anxiety and fat panic.
Stephanie Rodgers 