Towards Multi-Language Recipe Personalisation and Recommendation

Multi-language recipe personalisation and recommendation is an under-explored field of information retrieval in academic and production systems. The existing gaps in our current understanding are numerous, even on fundamental questions such as whether consistent and high-quality recipe recommendation can be delivered across languages. Motivated by this need, we consider the multi-language recipe recommendation setting and present grounding results that will help to establish the potential and absolute value of future work in this area. Our work draws on several billion events from millions of recipes, with published recipes and users incorporating several languages, including Arabic, English, Indonesian, Russian, and Spanish. We represent recipes using a combination of normalised ingredients, standardised skills and image embeddings obtained without human intervention. In modelling, we take a classical approach based on optimising an embedded bi-linear user-item metric space towards the interactions that most strongly elicit cooking intent. For users without interaction histories, a bespoke content-based cold-start model that predicts context and recipe affinity is introduced. We show that our approach to personalisation is stable and scales well to new languages. A robust cross-validation campaign is employed and consistently rejects baseline models and representations, strongly favouring those we propose. Our results are presented in a language-oriented (as opposed to model-oriented) fashion to emphasise the language-based goals of this work. We believe that this is the first large-scale work that evaluates the value and potential of multi-language recipe recommendation and personalisation.

[1]  Liliana Ferreira,et al.  Information Extraction from Unstructured Recipe Data , 2019, Proceedings of the 2019 5th International Conference on Computer and Technology Applications.

[2]  Mikhail Fain,et al.  Dividing and Conquering Cross-Modal Recipe Retrieval: from Nearest Neighbours Baselines to SoTA , 2019, ArXiv.

[3]  Marc Najork,et al.  Position Bias Estimation for Unbiased Learning to Rank in Personal Search , 2018, WSDM.

[4]  Wei Wang,et al.  Recommender system application developments: A survey , 2015, Decis. Support Syst..

[5]  Kevin Murphy,et al.  What’s Cookin’? Interpreting Cooking Videos using Text, Speech and Vision , 2015, NAACL.

[6]  GayGeri,et al.  Accurately Interpreting Clickthrough Data as Implicit Feedback , 2017 .

[7]  Steven C. H. Hoi,et al.  Learning Cross-Modal Embeddings With Adversarial Networks for Cooking Recipes and Food Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Jun Harashima,et al.  A Large-scale Recipe and Meal Data Collection as Infrastructure for Food Research , 2016, LREC.

[9]  Jason Weston,et al.  #TagSpace: Semantic Embeddings from Hashtags , 2014, EMNLP.

[10]  Jaewoo Kang,et al.  KitcheNette: Predicting and Ranking Food Ingredient Pairings using Siamese Neural Network , 2019 .

[11]  WangWei,et al.  Recommender system application developments , 2015 .

[12]  Shlomo Berkovsky,et al.  Recommending Food: Reasoning on Recipes and Ingredients , 2010, UMAP.

[13]  Alexander Felfernig,et al.  An overview of recommender systems in the healthy food domain , 2017, Journal of Intelligent Information Systems.

[14]  Thorsten Joachims,et al.  Accurately interpreting clickthrough data as implicit feedback , 2005, SIGIR '05.

[15]  Shuyang Li,et al.  Generating Personalized Recipes from Historical User Preferences , 2019, EMNLP.

[16]  Yoko Yamakata,et al.  English Recipe Flow Graph Corpus , 2020, LREC.

[17]  Damion M. Dooley,et al.  FoodOn: a harmonized food ontology to increase global food traceability, quality control and data integration , 2018, npj Science of Food.

[18]  Sebastian Bruch,et al.  TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank , 2018, KDD.

[19]  Christoph Trattner,et al.  An Evaluation of Recommendation Algorithms for Online Recipe Portals , 2019, HealthRecSys@RecSys.

[20]  Christoph Trattner,et al.  On the predictability of the popularity of online recipes , 2018, EPJ Data Science.

[21]  J. Bobadilla,et al.  Recommender systems survey , 2013, Knowl. Based Syst..

[22]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[23]  Keiji Yanai,et al.  [Demo paper] mirurecipe: A mobile cooking recipe recommendation system with food ingredient recognition , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[24]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[25]  Matthieu Cord,et al.  Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image Embeddings , 2018, SIGIR.

[26]  J. Mockus.  Bayesian Approach to Global Optimization: Theory and Applications , 1989 .

[27]  Amaia Salvador,et al.  Inverse Cooking: Recipe Generation From Food Images , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Amaia Salvador,et al.  Learning Cross-Modal Embeddings for Cooking Recipes and Food Images , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Christopher D. Manning,et al.  Stanza: A Python Natural Language Processing Toolkit for Many Human Languages , 2020, ACL.

[30]  Abdullah Al Mamun,et al.  Unsupervised Alignment of Actions in Video with Text Descriptions , 2016, IJCAI.

[31]  Chong-Wah Ngo,et al.  Deep Understanding of Cooking Procedure for Cross-modal Recipe Retrieval , 2018, ACM Multimedia.

[32]  Yi Tay,et al.  Deep Learning based Recommender System: A Survey and New Perspectives , 2018 .

[33]  Jianling Sun,et al.  MCEN: Bridging Cross-Modal Gap between Cooking Recipes and Dish Images with Latent Variable Model , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Paula Chocron,et al.  Vocabulary Alignment for Collaborative Agents: a Study with Real-World Multilingual How-to Instructions , 2018, IJCAI.

[35]  Lada A. Adamic,et al.  Recipe recommendation using ingredient networks , 2011, WebSci '12.

[36]  Mu Zhu,et al.  A Relationship between the Average Precision and the Area Under the ROC Curve , 2015, ICTIR.

[37]  M. Z. Rashad,et al.  Food Recommendation Using Ontology and Heuristics , 2012, AMLTA.

[38]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[40]  Yoko Yamakata,et al.  Categorization of Cooking Actions Based on Textual/Visual Similarity , 2019, MADiMa @ ACM Multimedia.

[41]  Dietmar Jannach,et al.  Are we really making much progress? A worrying analysis of recent neural recommendation approaches , 2019, RecSys.

[42]  Donghyeon Park,et al.  KitcheNette: Predicting and Recommending Food Ingredient Pairings using Siamese Neural Networks , 2019, IJCAI.

[43]  Rui Maia,et al.  Context-aware food recommendation system , 2018 .

[44]  Shlomo Berkovsky,et al.  Intelligent food planning: personalized recipe recommendation , 2010, IUI '10.

[45]  Maciej Kula,et al.  Metadata Embeddings for User and Item Cold-start Recommendations , 2015, CBRecSys@RecSys.