Can we trust GPT-4 Turbo Vision to estimate carbohydrate and caloric content from food images within the broad context of diabetes?

Lead Investigator: Joseph Cafazzo, Toronto General Hospitable Research Institute, University Health Network
Title of Proposal Research: Can we trust GPT-4 Turbo Vision to estimate carbohydrate and caloric content from food images within the broad context of diabetes?
Vivli Data Request: 9470
Funding Source: None
Potential Conflicts of Interest: None

Summary of the Proposed Research:

With 470 million people are affected by type 1 and type 2 diabetes and an additional 298 million people with prediabetes. We know deciding what to eat to support type 1 and type 2 diabetes management can be complicated. We also know diet changes like higher fiber diets drastically impact outcomes like improving “glycaemic control, blood lipids, weight, and inflammation”. We are also aware that there are too few clinicians and health coaches to support personalized medicine on a meal-to-meal basis. We want to explore the capabilities of GPT-4 Turbo with Vision as a “pocket coach”. But for it to be viable as part of a larger model of care, we need to know if we can trust the output. This study is about comparing estimates of carbohydrates and calories based on food photos from the pocket coach to macronutrient estimates by trained nutrition experts. The goal is to determine accuracy, and validity of the pocket coaches estimates. We ask the pocket coach to analyse the carbohydrate calorie content of food images and compare them to previous annotations by Pennington Biomedical Research Center. We will look both at the content of carbohydrate in grams and calories in kcals, as well as relative categories of carbohydrates (very low, low, moderate, and high carbohydrate content). Then we will test to see whether there is difference in how the pocket coach rates nutritional estimates, and explore under what conditions the pocket coach fails.

Requested Studies:

Type 1 Diabetes EXercise Initiative: The Effect of Exercise on Glycemic Control in Type 1 Diabetes Study
Data Contributor: Jaeb Center for Health Research Foundation, Inc.
Study ID: T1-DEXI
Sponsor ID: T1-DEXI

Type 1 Diabetes EXercise Initiative Pediatric Study (T1DexiP): The Effect of Exercise on Glycemic Control in Youth with Type 1 Diabetes
Data Contributor: Jaeb Center for Health Research Foundation, Inc.
Study ID: T1-DEXIP
Sponsor ID: T1-DEXIP