If you are looking for evidence of bad design in nutrition research, we have a fresh example to share with you.
A new study released last week compared two meals "equal" in calories and macronutrients among a group of men with type 2 diabetes. One meal was a pork cheeseburger with a latte containing 21 grams of sugar; the second meal was a veggie tofu burger with unsweetened green tea.
The researchers deemed this was a fair comparison between a "meat-based" meal and a "plant-based" meal to test blood markers for diabetes - despite one drink having 21 grams of sugar and the other drink having none. Strange, indeed, but isn't it interesting that they had to add that much sugar to the pork burger to bring the carbohydrate level up to be the same as the tofu burger?
The study was funded and conducted by the Physicians Committee for Responsible Medicine (PCRM), a vegan advocacy group. The bias of the PCRM is likely responsible for the blatantly uneven design of the study. And guess which meal was deemed by the researchers as achieving superior metabolic results for hormone markers for diabetes? That's right: the "plant-based" meal.
Yet, regardless of obvious bias, the study was published Feb 27 in the peer-reviewed journal Nutrients. It is distressing that such a flawed study can get through peer-review.
Nutrients: A plant-based meal stimulates incretin and insulin secretion more than an energy and macronutrient-matched standard meal in type 2 diabetes: A randomized crossover studyIn a press release heralding the study, lead author Hana Kahleova, MD, PhD, director of clinical research for PCRM, is quoted as saying:
The results add to the evidence that a plant-based diet should be considered a frontline treatment for type 2 diabetes.
And right on cue, the American Diabetes Association (ADA) issued a press release entitled:
ADA SmartBrief: Plant-based diet may benefit patients with diabetesBut is that what the study truly found? Can that kind of conclusion be made from this flawed design? Did this really examine the differences between meat and tofu on diabetic markers? Of course not.
As some have noted on Twitter, the results actually say "that in order to make a pork cheeseburger worse than a vegan burger, one would have to add a sugary drink."
The Twitter universe went wild with people calling the study biased, fraudulent and saying it should be retracted. Many noted that PCRM proudly proclaims its animal-rights agenda. In fact, its website notes its mission is dedicated to: "saving and improving human and animal lives through plant-based diets and ethical and effective scientific research."
Zooming out to the bigger picture, the most important takeaway from this compromised study is the reminder that too much of nutrition research is seriously flawed, with biased, poor designs. It shows that peer-review at journals is not always rigorous and does not necessarily raise questions about study design issues before studies are published. It shows, too, perhaps, that clear agendas rather than curiosity for the right answers drive research questions.
Alas, studies like this simply add to consumer confusion and create distorted headlines about the differences between plant and meat-based meals, without adding clarity or enlightenment.
We feel it is very important that the quality of nutrition research improve. It is also important that people interested in bettering their health through diet learn to interpret research studies with scepticism and to read the tables carefully in the studies to discern what exactly the researchers are comparing.
Helping our readers understand the strength and quality of research evidence is why Diet Doctor is working to evidence-base every one of our guides and has adopted a policy of how we grade scientific evidence. We hope these policies help you become discerning consumers of what purports to be nutrition news.
