Preprint
Article
Flawed Reanalysis Fails to Support the Carbohydrate-Insulin Model of Obesity
Altmetrics
Downloads
162
Views
97
Comments
0
A peer-reviewed article of this preprint also exists.
This version is not peer-reviewed
Submitted:
24 January 2024
Posted:
25 January 2024
You are already at the latest version
Alerts
Abstract
A recent reanalysis of publicly available data claimed that the carbohydrate-insulin model was supported by results from a randomized crossover trial comparing low carbohydrate (LC) versus low fat (LF) diets. The reanalysis also claimed to have invalidated the primary outcome of the trial, namely the within-participant differences in ad libitum energy intake when they consumed LC vs LF diets. Here, we demonstrate that both claims are untrue. The reanalysis was fatally flawed in several respects, with the most egregious errors related to ignoring the within-participant, repeated-measures structure of the data and treating data timepoints as independent. This resulted in almost all statistical comparisons being incorrect and reporting wildly optimistic p-values. Furthermore, the authors failed to engage sufficiently with prior work on the same trail data and portrayed their reanalysis as being more novel than it was. The reanalysis also failed to disclose the possibility of bias and succumbed to common statistical errors that falsely led the authors to interpret the data as supporting the carbohydrate-insulin model of obesity.
Keywords:
Subject: Biology and Life Sciences - Endocrinology and Metabolism
We read with great interest a recent article by Soto-Mota et al. [1] who presented secondary analyses of our random-order crossover study previously published in Nature Medicine [2]. The authors claim that our data supported the carbohydrate-insulin model of obesity [3,4,5,6]. This was surprising because the carbohydrate- insulin model predicts that high insulin secretion resulting from a high carbohydrate diet promotes increased body fat and increased ad libitum energy intake compared to a low carbohydrate diet – exactly the opposite of what occurred in our study [2]. Indeed, every single participant consumed fewer calories during the high carbohydrate, low fat (LF) diet and this occurred despite markedly higher insulin secretion and greater loss of body fat as compared to the ketogenic, low carbohydrate (LC) diet.
Soto-Mota et al. claimed to have undertaken their reanalysis of our data “to determine whether the primary findings [reported in our Nature Medicine paper] remain valid” when considering order effects recently reported by our group [7,8]. Unfortunately, Soto-Mota et al. failed to address the primary outcome of our study and did not acknowledge that there was no significant diet order effect on this primary outcome. Specifically, there was no significant diet order effect on the within-participant diet differences in ad libitum energy intake. Rather, Soto-Mota et al. ignored the within-participant design of our study and unjustifiably asserted that the differences between participants randomized to different diet order groups somehow invalidated our primary findings.
Readers may be unaware that most of the analyses presented by Soto-Mota et al. comparing the different diet order groups were not novel. Rather, they merely attempted to replicate our recently reported secondary analyses [7,8]. Indeed, Figure 1, Figure 2, and Figure 3A of Soto-Mota et al. convey the same information as Figure 2C, Figure 2E, and Figure 4A of our previously published pre-print [8] which is currently under revision at another journal. These results were also presented at a conference in July of 2023 [7].
Our previous analyses found that there were important energy balance effects between participants randomized to different diet order groups in our study, but we suggested vastly different interpretations and proposed mechanisms [8]. Unfortunately, these alternatives were completely ignored by Soto-Mota et al. For example, the diet order effect on ketone levels during the LC diet presented by Soto-Mota et al. in their Figure 3A was readily explained by differences in concurrent energy and carbohydrate intake. Additionally, small differences in respiratory quotient during the LF diet depicted in Figure 3B of Soto-Mota et al. were likely explained by the well-known effect of energy imbalance to affect the respiratory quotient [9].
Unlike our secondary analyses using the full dataset, Soto-Mota et al. used only a subset of data that were publicly available. The authors made no legitimate request for access to the non-public data which were not in the public database due to 4 participants withholding consent to broad data sharing. Unfortunately, Soto-Mota et al. failed to acknowledge that these non-public data were all coincidentally from participants randomized to the same diet order group. Therefore, their analyses included 40% fewer participants in one diet order group which had the potential to introduce significant and undisclosed bias.
Furthermore, the statistical analysis code of Soto-Mota et al. contains many technical flaws, with the most egregious errors related to ignoring the within-participant repeated measures structure of the data. For example, Soto-Mota et al. treated each data timepoint as if it were independent thereby overestimating the power of their analyses and making most of the p-values reported in their manuscript optimistically incorrect, including all p-values in their Figures 1–3.
One novel contribution of Soto-Mota et al. regards correlation plots presented in their Figure 4A,B used to support the carbohydrate-insulin model of obesity. These figures plot average energy intake (panel A) and change in body fat mass (panel B) in each individual participant during the second diet period against the change in fasting plasma C-peptide during the first diet period. Using the full data set, we plotted these variables in Figure 1 where we also indicated the diet order assignment of each participant. This figure highlights that the correlation analyses of Sota-Mota et al. succumbed to a common pitfall in statistical analysis where they failed to account for the fact that “the dataset has two subgroups of individuals whose values for one or both variables differ from each other [and] can lead to a false sense of relationship overall, even when none exists within each subgroup” [10]. Indeed, Figure 1 shows that there were no significant correlations between these variables within each diet order subgroup.
Sota-Mota et al. claimed that the spurious positive correlations presented in their Figure 4 corresponded to a prediction of the carbohydrate-insulin model of obesity because increased basal insulin secretion during the first diet period led to increased energy intake and body fat change in the second diet period. However, even if these correlations were real, this seems like an unlikely prediction of the carbohydrate-insulin model because it would additionally require that insulin secretion during the second diet period (which was significantly higher in the group with lower energy intake and fat mass change) was overshadowed by the effect of basal insulin secretion in the two weeks prior. Exactly how such a delayed effect of insulin could occur or is consistent with the carbohydrate-insulin model was unaddressed by Soto-Mota et al.
In summary, the paper by Sota-Mota et al. misleadingly suggested that many of their results were novel and failed to engage sufficiently with our prior work that proposed more likely mechanisms for the observed order effects between participants randomized to different diet order groups. Soto-Mota et al. ignored the within-participant design of our study, failed to disclose the possibility of bias, and committed numerous statistical errors in their fatally flawed reanalysis. Thus, despite the insistence of Soto-Mota et al. that our study’s primary findings were invalid, our data clearly demonstrated that ad libitum energy intake was lower when participants consumed the LF diet, independent of diet order, and this occurred despite markedly higher concurrent carbohydrate intake and insulin secretion. The machinations of Sota-Mota et al. do not change the fact that our study’s results were directly opposite to the predictions of the carbohydrate-insulin model.
Funding
This work was supported by the Intramural Research Program of the National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases.
Conflicts of Interest
None of the authors have financial conflicts of interest.
References
- Soto-Mota A, Jansen LT, Norwitz NG, Pereira MA, Ebbeling CB, Ludwig DS. Physiologic Adaptation to Macronutrient Change Distorts Findings from Short Dietary Trials: Reanalysis of a Metabolic Ward Study. J Nutr 2023. [CrossRef]
- Hall KD, Guo J, Courville AB, Boring J, Brychta R, Chen KY, Darcey V, Forde CG, Gharib AM, Gallagher I, et al. Effect of a plant-based, low-fat diet versus an animal-based, ketogenic diet on ad libitum energy intake. Nature medicine 2021;27(2):344-53. [CrossRef]
- Hall KD, Farooqi IS, Friedman JM, Klein S, Loos RJF, Mangelsdorf DJ, O'Rahilly S, Ravussin E, Redman LM, Ryan DH, et al. The energy balance model of obesity: Beyond calories in, calories out. Am J Clin Nutr 2022;115(5):1243-54. [CrossRef]
- Hall KD, Guyenet SJ, Leibel RL. The Carbohydrate-Insulin Model of Obesity Is Difficult to Reconcile With Current Evidence. JAMA internal medicine 2018;178(8):1103-5. [CrossRef]
- Ludwig DS, Aronne LJ, Astrup A, de Cabo R, Cantley LC, Friedman MI, Heymsfield SB, Johnson JD, King JC, Krauss RM, et al. The carbohydrate-insulin model: aAphysiological perspective on the obesity pandemic. Am J Clin Nutr 2021. [CrossRef]
- Ludwig DS, Ebbeling CB. The Carbohydrate-Insulin Model of Obesity: Beyond "Calories In, Calories Out". JAMA internal medicine 2018;178(8):1098-103. [CrossRef]
- Sciarrillo CM, Guo J, Hengist A, Darcey V, Hall KD. Diet Order Affects Energy Intake and Weight Change During a Crossover Low-carbohydrate vs. Low-fat Diet Study. Current developments in nutrition 2023;7:286. [CrossRef]
- Sciarrillo CM, Guo J, Hengist A, Darcey VL, Hall KD. Diet order affects energy balance in randomized crossover feeding studies that vary in macronutrients but not ultra-processing. medRxiv 2023. [CrossRef]
- Hall KD, Guo J, Chen KY, Leibel RL, Reitman ML, Rosenbaum M, Smith SR, Ravussin E. Methodologic considerations for measuring energy expenditure differences between diets varying in carbohydrate using the doubly labeled water method. Am J Clin Nutr 2019. [CrossRef]
- Aggarwal R, Ranganathan P. Common pitfalls in statistical analysis: The use of correlation techniques. Perspect Clin Res 2016;7(4):187-90. [CrossRef]
Figure 1.
Individual data showed no significant correlations within participants randomly assigned to the LC diet first (●) or LF diet first (○) groups between (A) average energy intake or (B) body fat change during phase 2 of a crossover study in relation to changes in fasting C-peptide during phase 1.
Figure 1.
Individual data showed no significant correlations within participants randomly assigned to the LC diet first (●) or LF diet first (○) groups between (A) average energy intake or (B) body fat change during phase 2 of a crossover study in relation to changes in fasting C-peptide during phase 1.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
MDPI Initiatives
Important Links
© 2024 MDPI (Basel, Switzerland) unless otherwise stated