Statistical oversight could explain inconsistencies in nutritional research

Thursday 13 Oct 2022

Filed under

The standard and most common statistical approach to studying the relationship between food and health can give misleading and meaningless results, according to new research published today in the American Journal of Clinical Nutrition.

The researchers, from The Alan Turing Institute and the University of Leeds, found that the widespread practice of statistically controlling, or allowing for, someone’s total energy intake can lead to dramatic changes in the interpretation of the results. 

People often wonder why one nutritional study tells them that eating too many eggs, for instance, will lead to heart disease and another tells them the opposite. This report shows that the answer to this and other conflicting food studies may lie in the use of statistics.

Lead author Georgia Tomova, a PhD researcher in the University of Leeds’ Institute for Data Analytics and The Alan Turing Institute, said: "These findings are relevant to everything we think we know about the effect of food on health. It is well known that different nutritional studies tend to find different results. One week a food is apparently harmful and the next week it is apparently good for you." 

Controlling for other foods eaten can then further skew the results, so that a harmful food appears beneficial or vice versa. 

Ms Tomova added: "Because of the big differences between individual studies, we tend to rely on review articles to provide an average estimate of whether, and to what extent, a particular food causes a particular health condition.  

“Unfortunately, because most studies have different approaches to controlling for the rest of the diet, it is likely that each study is estimating a very different quantity, making the 'average' rather meaningless.” 

The research, which was funded by The Alan Turing Institute, identified the problem by using new 'causal inference' methods, which were popularized by Judea Pearl, the author of “The Book of Why.” 

Senior author Dr Peter Tennant, Associate Professor of Health Data Science in Leeds’ School of Medicine and Turing Fellow, said: "When you cannot run an experiment, it is very difficult to determine whether, and to what extent, something causes something else.  

“That is why people say, 'correlation does not equal causation.' These new 'causal inference' methods promise to help us to identify causal effects from correlations, but in doing so they have also highlighted quite a few areas which we did not fully understand." 

The authors hope that this new research will help nutritional scientists to better understand the issues with inappropriately controlling for total energy intake and overall diet and gain a clearer understanding of the effects of the diet on health. 

Dr Tennant added: "Different studies can provide different estimates for a range of reasons but we think that this one statistical issue may explain a lot of the disagreement. Fortunately, this can be easily avoided in the future.” 

Top image credit: Ella Olsson