Bad Research

In May, The New York Times posted an article about a research study that found that it’s possible to lose 11 pounds in four days. I’m disappointed in The Times for featuring this study, and if you’re a budding researcher or at all interested in debunking weight-loss myths, follow along with me as to why.

First, the study consisted of 15 participants, an embarrassingly low sample size. Let’s just say we can overlook that fact for a second. The article reports that the participants were “healthy but overweight Swedish men.” Ask yourself: “Am I a healthy but overweight Swedish man?” If not, the sample doesn’t represent you.

You’ll also notice that the sample doesn’t represent women, Swedish men of different health- and weight- statuses, non-Swedish men – you get the picture. It’s a homogenous sample, and that’s not the goal of good research.

Next, let’s take a look at the experimental condition (note: there was just one condition and not a control group, further decreasing research validity). During the study, participants ate about 360 calories a day and exercised for almost nine hours. Like The Biggest Loser on steroids. . . When we think of research, we like to choose conditions that can be replicated in the real world, to increase the study’s internal validity. How likely is it that most people could drop down to such a caloric deficit and exercise for so many hours in their “real lives?” If not, then the results we see could somehow be an artifact of the experimental conditions.

Finally, let’s talk about the conditions themselves. To repeat, these men consumed approximately 360 calories a day and exercised for almost nine hours. This sounds more disordered than some of the most serious eating disorders I’ve encountered. Will these men go on to develop a disordered relationship with food or exercise? Who knows? But, I’m not sure where we draw the line with ethical research and what type of research review board approved this study’s intervention. I could argue that infecting research participants with some sort of food poisoning could also result in such rapid and significant weight-loss, but that would obviously be unhealthy. But would it be any more unhealthy than what these researchers prescribed?

And if we’re to generalize from this research and start prescribing a few days of extreme caloric restriction and over-exercise to treat those who are overweight, do we want to run the risk of some patients going on to develop disordered eating? Did the researchers look at the participants’ attitudes toward food and weight as a result of the intervention? I’m reminded of the Minnesota Starvation Experiment. What happens after a year? Is weight loss really the ultimate goal?