Food Recommendation: Framework, Existing Solutions, and Challenges

A growing proportion of the global population is becoming overweight or obese, leading to various diseases (e.g., diabetes, ischemic heart disease and even cancer) due to unhealthy eating patterns, such as increased intake of food with high energy and high fat. Food recommendation is of paramount importance to alleviate this problem. Unfortunately, modern multimedia research has enhanced the performance and experience of multimedia recommendation in many fields such as movies and POI, yet largely lags in the food domain. This article proposes a unified framework for food recommendation, and identifies main issues affecting food recommendation including incorporating various context and domain knowledge, building the personal model, and analyzing unique food characteristics. We then review existing solutions for these issues, and finally elaborate research challenges and future directions in this field. To our knowledge, this is the first survey that targets the study of food recommendation in the multimedia field and offers a collection of research studies and technologies to benefit researchers in this field.

[2]  Wei-Ta Chu,et al.  A hybrid recommendation system considering visual information for predicting favorite restaurants , 2017, World Wide Web.

[3]  Lili Wang,et al.  Recent Progress of Self-Powered Sensing Systems for Wearable Electronics. , 2017, Small.

[4]  Yiu-Kai Ng,et al.  Personalized Recipe Recommendations for Toddlers Based on Nutrient Intake and Food Preferences , 2017, MEDES.

[5]  Amaia Salvador,et al.  Learning Cross-Modal Embeddings for Cooking Recipes and Food Images , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Chong-Wah Ngo,et al.  Deep-based Ingredient Recognition for Cooking Recipe Retrieval , 2016, ACM Multimedia.

[7]  Christoph Trattner,et al.  The Impact of Recipe Features, Social Cues and Demographics on Estimating the Healthiness of Online Recipes , 2018, ICWSM.

[8]  Ramesh C. Jain,et al.  Live Personalized Nutrition Recommendation Engine , 2017, MMHealth@MM.

[9]  Tat-Seng Chua,et al.  Hierarchical Attention Network for Visually-Aware Food Recommendation , 2018, IEEE Transactions on Multimedia.

[10]  Ramesh C. Jain,et al.  A Survey on Food Computing , 2018, ACM Comput. Surv..

[11]  Sergio Guadarrama,et al.  Im2Calories: Towards an Automated Mobile Vision Food Diary , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Shlomo Berkovsky,et al.  Intelligent food planning: personalized recipe recommendation , 2010, IUI '10.

[13]  James Fogarty,et al.  Rethinking the Mobile Food Journal: Exploring Opportunities for Lightweight Photo-Based Capture , 2015, CHI.

[14]  Susanne Boll,et al.  Digital Health Devices for Everyone! , 2014, IEEE Pervasive Computing.

[15]  Keiji Yanai,et al.  Real-time mobile recipe recommendation system using food ingredient recognition , 2012, IMMPD '12.

[16]  Yong Rui,et al.  You Are What You Eat: Exploring Rich Recipe Information for Cross-Region Food Analysis , 2018, IEEE Transactions on Multimedia.

[17]  Lada A. Adamic,et al.  Recipe recommendation using ingredient networks , 2011, WebSci '12.

[18]  Hui Xiong,et al.  User Preference Learning with Multiple Information Fusion for Restaurant Recommendation , 2014, SDM.

[19]  Shuqiang Jiang,et al.  Multi-Scale Multi-View Deep Feature Aggregation for Food Recognition , 2020, IEEE Transactions on Image Processing.

[20]  Martina Ziefle,et al.  Information and Communication Technologies for Ageing Well and e-Health , 2017, Communications in Computer and Information Science.

[21]  Mouzhi Ge,et al.  Using Tags and Latent Factors in a Food Recommender System , 2015, Digital Health.

[22]  Diego López-de-Ipiña,et al.  Enhancing Profile and Context Aware Relevant Food Search through Knowledge Graphs , 2018, UCAmI.

[23]  Kiyoharu Aizawa,et al.  Personalized Classifier for Food Image Recognition , 2018, IEEE Transactions on Multimedia.

[24]  Chong-Wah Ngo,et al.  Cross-modal Recipe Retrieval with Rich Food Attributes , 2017, ACM Multimedia.

[25]  Xiangyang Li,et al.  Where and What to Eat: Simultaneous Restaurant and Dish Recognition from Food Image , 2016, PCM.

[26]  Minyi Guo,et al.  DKN: Deep Knowledge-Aware Network for News Recommendation , 2018, WWW.

[27]  Matthieu Guillaumin,et al.  Food-101 - Mining Discriminative Components with Random Forests , 2014, ECCV.

[28]  Ramesh C. Jain,et al.  Cross-Modal Health State Estimation , 2018, ACM Multimedia.

[29]  Ramesh C. Jain,et al.  Objective Self , 2014, IEEE Multim..

[30]  Noel E. O'Connor,et al.  Health Media: From Multimedia Signals to Personal Health Insights , 2018, IEEE MultiMedia.

[31]  J. Verhagen,et al.  The neurocognitive bases of human multimodal food perception: Sensory integration , 2006, Neuroscience & Biobehavioral Reviews.

[32]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[33]  Deborah Estrin,et al.  Yum-Me: A Personalized Nutrient-Based Meal Recommender System , 2016, ACM Trans. Inf. Syst..

[34]  David Ribeiro,et al.  SousChef: Improved Meal Recommender System for Portuguese Older Adults , 2017, ICT4AgeingWell.

[35]  Mouzhi Ge,et al.  Health-aware Food Recommender System , 2015, RecSys.

[36]  Shuang Wang,et al.  Geolocalized Modeling for Dish Recognition , 2015, IEEE Transactions on Multimedia.

[37]  Seunghyeon Kim,et al.  Nutritionally recommended food for semi- to strict vegetarian diets based on large-scale nutrient composition data , 2018, Scientific Reports.

[38]  Mu Zhu,et al.  Content-boosted matrix factorization for recommender systems: experiments with recipe recommendation , 2011, RecSys '11.

[39]  Bernd Ludwig,et al.  User Nutrition Modelling and Recommendation: Balancing Simplicity and Complexity , 2017, UMAP.

[40]  Marie Katsurai,et al.  Recipe Popularity Prediction with Deep Visual-Semantic Fusion , 2017, CIKM.

[41]  Keiji Yanai,et al.  [Demo paper] mirurecipe: A mobile cooking recipe recommendation system with food ingredient recognition , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[42]  Ramesh C. Jain,et al.  Multimodal Food Journaling , 2018, HealthMedia@MM.

[43]  Christoph Trattner,et al.  Exploiting Food Choice Biases for Healthier Recipe Recommendation , 2017, SIGIR.

[44]  Christoph Trattner,et al.  Food Recommender Systems: Important Contributions, Challenges and Future Research Directions , 2017, ArXiv.

[45]  Michael Gasser,et al.  The Development of Embodied Cognition: Six Lessons from Babies , 2005, Artificial Life.

[46]  Valeria De Antonellis,et al.  A Web-Based Application for Semantic-Driven Food Recommendation with Reference Prescriptions , 2015, WISE.

[47]  David Elsweiler,et al.  Towards Automatic Meal Plan Recommendations for Balanced Nutrition , 2015, RecSys.

[48]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[49]  Takahira Yamaguchi,et al.  Eliciting User Food Preferences in terms of Taste and Texture in Spoken Dialogue Systems , 2018, MHFI@ICMI.

[50]  Lu Qi,et al.  The short-chain fatty acid propionate increases glucagon and FABP4 production, impairing insulin action in mice and humans , 2019, Science Translational Medicine.

[51]  Ramesh C. Jain,et al.  Health Multimedia: Lifestyle Recommendations Based on Diverse Observations , 2017, ICMR.

[52]  A. Drewnowski,et al.  Development and validation of the nutrient-rich foods index: a tool to measure nutritional quality of foods. , 2009, The Journal of nutrition.

[53]  Fan Zhang,et al.  PlateClick: Bootstrapping Food Preferences Through an Adaptive Visual Interface , 2015, CIKM.

[54]  Marios Anthimopoulos,et al.  Two-View 3D Reconstruction for Food Volume Estimation , 2017, IEEE Transactions on Multimedia.

[55]  Luis Herranz,et al.  Being a Supercook: Joint Food Attributes and Multimodal Content Modeling for Recipe Retrieval and Exploration , 2017, IEEE Transactions on Multimedia.

[56]  Eliza Strickland 3 sensors to track every bite and gulp [News] , 2018 .

[57]  Kiyoharu Aizawa,et al.  Food Balance Estimation by Using Personal Dietary Tendencies in a Multimedia Food Log , 2013, IEEE Transactions on Multimedia.

[58]  Shuqiang Jiang,et al.  Ingredient-Guided Cascaded Multi-Attention Network for Food Recognition , 2019, ACM Multimedia.

[59]  T. Vos,et al.  Global, regional, and national incidence and prevalence, and years lived with disability for 328 diseases and injuries in 195 countries, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016 , 2017 .

[60]  Shou-De Lin,et al.  A Content-Based Matrix Factorization Model for Recipe Recommendation , 2014, PAKDD.

[61]  Ramesh C. Jain,et al.  Pocket Dietitian: Automated Healthy Dish Recommendations by Location , 2017, ICIAP Workshops.

[62]  Shuqiang Jiang,et al.  A Delicious Recipe Analysis Framework for Exploring Multi-Modal Recipes with Various Attributes , 2017, ACM Multimedia.

[63]  Chong-Wah Ngo,et al.  Food Photo Recognition for Dietary Tracking: System and Experiment , 2018, MMM.

[64]  Sandrine Péneau,et al.  Validation of the FSA nutrient profiling system dietary index in French adults—findings from SUVIMAX study , 2016, European Journal of Nutrition.

[65]  Kiyoharu Aizawa,et al.  FoodLog: Multimedia Tool for Healthcare Applications , 2015, IEEE MultiMedia.

[66]  Julien Delarue,et al.  Dynamics of food preferences: a case study with chewing gums , 2004 .

[67]  Luis Herranz,et al.  A survey on context-aware mobile visual recognition , 2017, Multimedia Systems.