Market2Dish: Health-aware Food Recommendation

With the rising incidence of some diseases, such as obesity and diabetes, the healthy diet is arousing increasing attention. However, most existing food-related research efforts focus on recipe retrieval, user-preference-based food recommendation, cooking assistance, or the nutrition and calorie estimation of dishes, ignoring the personalized health-aware food recommendation. Therefore, in this work, we present a personalized health-aware food recommendation scheme, namely, Market2Dish, mapping the ingredients displayed in the market to the healthy dishes eaten at home. The proposed scheme comprises three components, namely, recipe retrieval, user health profiling, and health-aware food recommendation. In particular, recipe retrieval aims to acquire the ingredients available to the users and then retrieve recipe candidates from a large-scale recipe dataset. User health profiling is to characterize the health conditions of users by capturing the textual health-related information crawled from social networks. Specifically, to solve the issue that the health-related information is extremely sparse, we incorporate a word-class interaction mechanism into the proposed deep model to learn the fine-grained correlations between the textual tweets and pre-defined health concepts. For the health-aware food recommendation, we present a novel category-aware hierarchical memory network–based recommender to learn the health-aware user-recipe interactions for better food recommendation. Moreover, extensive experiments demonstrate the effectiveness of the health-aware food recommendation scheme.

[1]  Tat-Seng Chua,et al.  Neural Graph Collaborative Filtering , 2019, SIGIR.

[2]  Shuohang Wang,et al.  Learning Natural Language Inference with LSTM , 2015, NAACL.

[3]  Munmun De Choudhury,et al.  Characterizing Dietary Choices, Nutrition, and Language in Food Deserts via Social Media , 2016, CSCW.

[4]  Hala Skaf-Molli,et al.  TAAABLE 3: Adaptation of ingredient quantities and of textual preparations , 2010, ICCBR 2010.

[5]  Xiangnan He,et al.  A Generic Coordinate Descent Framework for Learning from Implicit Feedback , 2016, WWW.

[6]  Lior Wolf,et al.  Using the Output Embedding to Improve Language Models , 2016, EACL.

[7]  Kiyoharu Aizawa,et al.  Food log by analyzing food images , 2008, ACM Multimedia.

[8]  Michele Merler,et al.  Learning to Make Better Mistakes: Semantics-aware Visual Food Recognition , 2016, ACM Multimedia.

[9]  Jialie Shen,et al.  Just-for-Me: An Adaptive Personalization System for Location-Aware Social Music Recommendation , 2014, ICMR.

[10]  Yueting Zhuang,et al.  User Preference Learning for Online Social Recommendation , 2016, IEEE Transactions on Knowledge and Data Engineering.

[11]  Hanwang Zhang,et al.  "Click" Is Not Equal to "Like": Counterfactual Recommendation for Mitigating Clickbait Issue , 2020, ArXiv.

[12]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[13]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[14]  Fumin Shen,et al.  Chat More: Deepening and Widening the Chatting Topic via A Deep Model , 2018, SIGIR.

[15]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[16]  Antonio Torralba,et al.  Is Saki #delicious?: The Food Perception Gap on Instagram and Its Relation to Health , 2017, WWW.

[17]  Hamed Haddadi,et al.  #FoodPorn: Obesity Patterns in Culinary Interactions , 2015, Digital Health.

[18]  Yu Yang,et al.  Substructure similarity measurement in chinese recipes , 2008, WWW.

[19]  Chaoran Cui,et al.  Routing Micro-videos via A Temporal Graph-guided Recommendation System , 2019, ACM Multimedia.

[20]  Yiming Yang,et al.  An Evaluation of Statistical Approaches to Text Categorization , 1999, Information Retrieval.

[21]  Jun Harashima,et al.  Cookpad Image Dataset: An Image Collection as Infrastructure for Food Research , 2017, SIGIR.

[22]  John Riedl,et al.  An Algorithmic Framework for Performing Collaborative Filtering , 1999, SIGIR Forum.

[23]  Oren Barkan,et al.  ITEM2VEC: Neural item embedding for collaborative filtering , 2016, 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP).

[24]  Yiqun Liu,et al.  Learning on Partial-Order Hypergraphs , 2018, WWW.

[25]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[26]  Hiroshi Murase,et al.  Finding replaceable materials in cooking recipe texts considering characteristic cooking actions , 2009, CEA '09.

[27]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[28]  Yong Rui,et al.  You Are What You Eat: Exploring Rich Recipe Information for Cross-Region Food Analysis , 2018, IEEE Transactions on Multimedia.

[29]  Ling-Yu Duan,et al.  Market2Dish: A Health-aware Food Recommendation System , 2019, ACM Multimedia.

[30]  Christoph Trattner,et al.  Investigating the Healthiness of Internet-Sourced Recipes: Implications for Meal Planning and Recommender Systems , 2017, WWW.

[31]  Bernd Ludwig,et al.  You Are What You Eat: Learning User Tastes for Rating Prediction , 2013, SPIRE.

[32]  Sofiane Abbar,et al.  You Tweet What You Eat: Studying Food Consumption Through Twitter , 2014, CHI.

[33]  Elisa Bertino,et al.  Privacy-Preserving User Profile Matching in Social Networks , 2020, IEEE Transactions on Knowledge and Data Engineering.

[34]  Christoph Trattner,et al.  Temporality in Online Food Recipe Consumption and Production , 2015, WWW.

[35]  Christoph Trattner,et al.  The Impact of Recipe Features, Social Cues and Demographics on Estimating the Healthiness of Online Recipes , 2018, ICWSM.

[36]  Makoto Ogawa,et al.  Food Detection and Recognition Using Convolutional Neural Network , 2014, ACM Multimedia.

[37]  Tat-Seng Chua,et al.  Graph Adversarial Training: Dynamically Regularizing Based on Graph Structure , 2019, IEEE Transactions on Knowledge and Data Engineering.

[38]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Qi Tian,et al.  Multimodal Dialog System: Generating Responses via Adaptive Decoders , 2019, ACM Multimedia.

[40]  Chong-Wah Ngo,et al.  Cross-modal Recipe Retrieval with Rich Food Attributes , 2017, ACM Multimedia.

[41]  Tat-Seng Chua,et al.  Tweet Can Be Fit , 2017, ACM Trans. Inf. Syst..

[42]  Tat-Seng Chua,et al.  Fast Matrix Factorization for Online Recommendation with Implicit Feedback , 2016, SIGIR.

[43]  Mei Chen,et al.  Food recognition using statistics of pairwise local features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Tian Gan,et al.  Explicit Interaction Model towards Text Classification , 2018, AAAI.

[45]  Chong-Wah Ngo,et al.  Cross-Modal Recipe Retrieval: How to Cook this Dish? , 2017, MMM.

[46]  Plamen P. Angelov,et al.  Creating Evolving User Behavior Profiles Automatically , 2012, IEEE Transactions on Knowledge and Data Engineering.

[47]  Shlomo Berkovsky,et al.  Intelligent food planning: personalized recipe recommendation , 2010, IUI '10.

[48]  Marie Katsurai,et al.  Recipe Popularity Prediction with Deep Visual-Semantic Fusion , 2017, CIKM.

[49]  Shervin Shirmohammadi,et al.  Mobile Multi-Food Recognition Using Deep Learning , 2017, ACM Trans. Multim. Comput. Commun. Appl..

[50]  Christoph Trattner,et al.  Food Recommender Systems: Important Contributions, Challenges and Future Research Directions , 2017, ArXiv.

[51]  Chong-Wah Ngo,et al.  Deep Understanding of Cooking Procedure for Cross-modal Recipe Retrieval , 2018, ACM Multimedia.

[52]  Shuqiang Jiang,et al.  A Delicious Recipe Analysis Framework for Exploring Multi-Modal Recipes with Various Attributes , 2017, ACM Multimedia.

[53]  Shuqiang Jiang,et al.  Ingredient-Guided Cascaded Multi-Attention Network for Food Recognition , 2019, ACM Multimedia.

[54]  Christoph Trattner,et al.  Exploiting Food Choice Biases for Healthier Recipe Recommendation , 2017, SIGIR.

[55]  Matthieu Cord,et al.  Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image Embeddings , 2018, SIGIR.

[56]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[57]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[58]  Mouzhi Ge,et al.  Using Tags and Latent Factors in a Food Recommender System , 2015, Digital Health.

[59]  Amaia Salvador,et al.  Learning Cross-Modal Embeddings for Cooking Recipes and Food Images , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Mouzhi Ge,et al.  Health-aware Food Recommender System , 2015, RecSys.

[61]  Jialie Shen,et al.  On Effective Location-Aware Music Recommendation , 2016, ACM Trans. Inf. Syst..

[62]  Tat-Seng Chua,et al.  Denoising Implicit Feedback for Recommendation , 2020, WSDM.

[63]  Huanbo Luan,et al.  PIC2DISH: A Customized Cooking Assistant System , 2017, ACM Multimedia.

[64]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.