From Research Lab to Your Pocket: The MyFoodRepo Story

Hello from the MyFoodRepo team! We’ve been busy building MyFoodRepo and making it available to the public. I’d like to explain briefly where MyFoodRepo comes from, and what we’re trying to achieve.

MyFoodRepo started around 2017 as a necessity. At the time, we were planning a new study, Food & You; and we needed a tool to help participants track nutrition. Since the idea was to have both diet data and glucose response data in real time, we couldn’t use standard tools like food frequency questionnaires, or 24 hour recall. We needed to know what people were eating, and when, during a 14 day period.

We knew that only way to do this would be through an app. And we also knew that AI would be a key part in the solution (we had worked on image recognition in other contexts, so we knew about the potential of AI for food recognition early on). However, there were no good apps at the time, so we built our own.

Enter AI

In the beginning, we realised that AI would be extremely helpful. But we also understood that the quality wasn’t quite there yet for scientific standards. We had two options: let users check themselves and fix the mistakes AI made; or do it ourselves.

We rapidly chose the second option. The reason was that we couldn’t really afford people dropping off from the study en masse just because food tracking was hard! So we built a system where AI would do the initial step, and then human annotators in our research group would check and fix mistakes. Only once they approved, would the information appear on a user’s phone.
 
This system worked well - but of course it did not scale. That is why we were never able to make the app available to a larger public. We started investing a lot in making our AI better, even going so far as to develop our own food recognition benchmark.

The World Post-ChatGPT

Fast forward a few years, and ChatGPT comes out. When OpenAI first rolled out the ability to analyze images, I immediately uploaded an image of food. I was instantly blown away by how good the model was in its response. It immediately became clear that these models would be much better than anything we could ever come up with. So we began rewriting the entire application, with the goal of using the rapidly advancing AI models by the frontier labs.

Despite initial hallucinations, we had a hunch that the models would rapidly improve, and get cheaper. This bet paid off. The models are still not perfect, but the error rate is now so low that it’s perfectly fine to ask the users to correct any remaining mistake themselves. And indeed, thanks to AI, you can now correct issue with natural language! With this approach, we were now ready to scale, and thus we launched myfoodrepo.ai.

As the models get better, MyFoodRepo will now get better along with them! In the meanwhile, we’re working on many new features, such as the calculation of the healthy eating index (HEI), or of the integration of a system-wide chatbot. 

So what does this mean for anyone trying to track their nutrition today? MyFoodRepo combines the convenience of AI-powered food recognition with the accuracy needed for real health insights. Simply snap a photo of your meal, and within seconds you'll see a detailed nutritional breakdown. If the AI gets something wrong (maybe it thinks your quinoa is rice), just tell it in plain English, or any language you want: "That's actually quinoa, not rice" - and it corrects instantly.

MyFoodRepo understands food the way you do, visually and contextually. It can recognize complex meals, estimate portions, and even handle that homemade stir-fry that would be impossible to log traditionally.

Imagine you're having lunch at your favorite local restaurant. You snap a photo of your salmon bowl. MyFoodRepo instantly recognizes the grilled salmon, brown rice, avocado, mixed greens, and sesame dressing - calculating not just calories but comprehensive nutrition data including macros, vitamins, and minerals. If it misses the pine nuts sprinkled on top, just type "add pine nuts" and you're done. No searching through databases, no guessing portions.

We're incredibly excited about what's ahead. The healthy eating index (HEI) feature, which we will launch soon, will help you understand not just what you're eating, but how well your diet aligns with nutritional guidelines. Further down the line, the MyFoodRepo chatbot will be able to answer questions like "How much protein have I had this week?" or "What should I eat to hit my iron goals today?"

Want to experience the future of nutrition tracking? Subscribe at myfoodrepo.ai, download the app, and start today. We'd love to hear about your experience - because at the end of the day, we want to make healthy eating easier for everyone.

To your health,
Marcel Salathé