AI can suggest recipes from photos of food

Pixabay|Pixabay|Pixabay

Given a photo of food, a new deep-learning algorithm can recommend ingredients and recipes that match.

Artificial intelligence (AI) has improved enough to analyze medical scans and even offer blessings in lieu of a priest, now researchers from MIT’s Computer Science and Artificial Intelligence Laboratory, and the Qatar Computing Research Institute have trained an AI system called Pic2Recipe to analyze photos of food and predict the ingredients used.

Up to now, the problem with training AI to analyse food has been a lack of large enough data sets. This problem has been exacerbated by the vast range of foods throughout the world. For example, a food database developed at City University in Hong Kong contained more than 110,000 images and 65,000 recipes, but only covered Chinese cuisine. The team at MIT has gotten around this problem by using photos on social media. Researchers combed websites like Food.com to create a database containing more than 1 million recipes. They then used that data to train a neural network to find patterns and make connections between the food images and the corresponding ingredients and recipes. The network can identify ingredients and suggest similar recipes from the database (people can also upload their own food photos to test it out).

The system still has several drawbacks. While it does very well with desserts like cookies and cakes, it is often stumped by dishes that have many different recipes – such as lasagna (e.g. with meat, vegetarian, vegan, etc). Once researchers have worked out these issues, they hope that the system could eventually be used to help people track their daily nutrition, determine the nutritional information of food, and even to duplicate restaurant meals at home later. The researchers are also interested in potentially developing the system into a “dinner aide” that could figure out what to cook given a dietary preference and a list of items in the fridge. What other uses might there be for a recipe AI?