A decision-analytic framework for interpretable recommendation systems with multiple input data sources: a case study for a European e-tailer

Recommendation systems help companies construct online personalization strategies for customers who are often overwhelmed by the abundance of product choices available. To extend existing operations research literature on recommendation systems, this article proposes a decision analytic framework for interpretable recommendation systems with multiple input data sources for e-commerce settings. The impact of multiple data sources on recommendation performance is investigated and two hybridization data fusion strategies, i.e., a posteriori weighting and input data source combination using factorization machines are benchmarked. Furthermore, a new importance score mechanism is introduced to provide insight into the input data sources’ and underlying variables’ impact on recommendation performance. The framework is empirically validated on 164,338 customers and 51,367 products across eight real-life data sets with four input data sources (product, customer, raw behavioral, and aggregated behavioral data) obtained from a large European e-commerce company. With this new decision analytic framework, e-commerce companies are able to open their recommendation system’s black box, to identify the most predictive input data sources and the best hybridization strategy for their business context.

[1]  Bart Baesens,et al.  Comprehensible Credit Scoring Models Using Rule Extraction from Support Vector Machines , 2007, Eur. J. Oper. Res..

[2]  Gerhard Friedrich,et al.  Recommender Systems: RECENT DEVELOPMENTS , 2010 .

[3]  Bart Baesens,et al.  Using Neural Network Rule Extraction and Decision Tables for Credit - Risk Evaluation , 2003, Manag. Sci..

[4]  Bart Baesens,et al.  New Insights into Churn Prediction in the Telco Sector , 2012 .

[5]  Dilip Singh Sisodia,et al.  Pair-wise Preference Relation based Probabilistic Matrix Factorization for Collaborative Filtering in Recommender System , 2020, Knowl. Based Syst..

[6]  Toly Chen,et al.  Integer nonlinear programming and optimized weighted-average approach for mobile hotel recommendation by considering travelers’ unknown preferences , 2018, Oper. Res..

[7]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[8]  Feng Zhou,et al.  EMD2FNN: A strategy combining empirical mode decomposition and factorization machine based neural network for stock market trend prediction , 2019, Expert Syst. Appl..

[9]  Aimee Elizabeth Taylor.  Statistical enhancement of support vector machines , 2009 .

[10]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[11]  Kristof Coussement,et al.  A framework for configuring collaborative filtering-based recommendations derived from purchase data , 2018, Eur. J. Oper. Res..

[12]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[13]  Charles Elkan,et al.  Optimal Thresholding of Classifiers to Maximize F1 Measure , 2014, ECML/PKDD.

[14]  Michael Scholz,et al.  A configuration-based recommender system for supporting e-commerce decisions , 2017, Eur. J. Oper. Res..

[15]  Wen Zhang,et al.  DeepRec: A deep neural network approach to recommendation with item embedding and weighted loss function , 2019, Inf. Sci..

[16]  Bart Baesens,et al.  New insights into churn prediction in the telecommunication sector: A profit driven data mining approach , 2012, Eur. J. Oper. Res..

[17]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[18]  Amir Albadvi,et al.  Integrating rating-based collaborative filtering with customer lifetime value: New product recommendation technique , 2010, Intell. Data Anal..

[19]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[20]  Stefan Lessmann,et al.  Incorporating textual information in customer churn prediction models based on a convolutional neural network , 2019 .

[21]  Koby Crammer,et al.  Breaking the curse of kernelization: budgeted stochastic gradient descent for large-scale SVM training , 2012, J. Mach. Learn. Res..

[22]  Konstantinos Bougiatiotis,et al.  Enhanced movie content similarity based on textual, auditory and visual information , 2017, Expert Syst. Appl..

[23]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[24]  Foster J. Provost,et al.  Mining Massive Fine-Grained Behavior Data to Improve Predictive Analytics , 2016, MIS Q..

[25]  George Karypis,et al.  Item-based top-N recommendation algorithms , 2004, TOIS.

[26]  Qinghua Liu,et al.  Diverse personalized recommendations with uncertainty from implicit preference data with the Bayesian Mallows model , 2019, Knowl. Based Syst..

[27]  Mohammad S. Rahman,et al.  Technology Usage and Online Sales: An Empirical Study , 2010, Manag. Sci..

[28]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[29]  Prakash P. Shenoy,et al.  A new heuristic for learning Bayesian networks from limited datasets: a real-time recommendation system application with RFID systems in grocery stores , 2012, Annals of Operations Research.

[30]  Gerhard Friedrich,et al.  Recommender Systems - An Introduction , 2010 .

[31]  Michael J. Pazzani,et al.  A Framework for Collaborative, Content-Based and Demographic Filtering , 1999, Artificial Intelligence Review.

[32]  Wu Zhijun,et al.  Low-Rate DDoS Attack Detection Based on Factorization Machine in Software Defined Network , 2020, IEEE Access.

[33]  Benjamin T. Hazen,et al.  Embedded analytics: improving decision support for humanitarian logistics operations , 2019, Ann. Oper. Res..

[34]  Yuan-Chun Jiang,et al.  Maximizing customer satisfaction through an online recommendation system: A novel associative classification model , 2010, Decis. Support Syst..

[35]  Bart Baesens,et al.  Development and application of consumer credit scoring models using profit-based classification measures , 2014, Eur. J. Oper. Res..

[36]  Simon Dooms,et al.  Dynamic generation of personalized hybrid recommender systems , 2013, RecSys.

[37]  Richard Vidgen,et al.  Management challenges in creating value from business analytics , 2017, Eur. J. Oper. Res..

[38]  Qiang Qu,et al.  Contextual-boosted deep neural collaborative filtering model for interpretable recommendation , 2019, Expert Syst. Appl..

[39]  Panagiota Galetsi,et al.  A review of the literature on big data analytics in healthcare , 2019, J. Oper. Res. Soc..

[40]  Bart Baesens,et al.  Social network analytics for churn prediction in telco: Model building, evaluation and network architecture , 2017, Expert Syst. Appl..

[41]  Simon Dooms,et al.  Recommender systems challenge 2014 , 2014, RecSys '14.

[42]  Pradeep Kumar,et al.  Recommendation generation using personalized weight of meta-paths in heterogeneous information networks , 2020, Eur. J. Oper. Res..

[43]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[44]  Mohammad Yahya H. Al-Shamri,et al.  User profiling approaches for demographic recommender systems , 2016, Knowl. Based Syst..

[45]  Dietmar Jannach,et al.  Recommender Systems by Dietmar Jannach , 2010 .

[46]  Il-Yeol Song,et al.  Database Design for Real-World E-Commerce Systems. , 2000 .

[47]  J. Bobadilla,et al.  Recommender systems survey , 2013, Knowl. Based Syst..

[48]  Yichuan Tang,et al.  Deep Learning using Linear Support Vector Machines , 2013, 1306.0239.

[49]  Majd Kharfan,et al.  A data-driven forecasting approach for newly launched seasonal products by leveraging machine-learning approaches , 2020, Annals of Operations Research.

[50]  Martin Kunc,et al.  Business analytics: Defining the field and identifying a research agenda , 2020, Eur. J. Oper. Res..

[51]  John Riedl,et al.  Analysis of recommendation algorithms for e-commerce , 2000, EC '00.

[52]  Bin Ran,et al.  Day-ahead traffic flow forecasting based on a deep belief network optimized by the multi-objective particle swarm algorithm , 2019, Knowl. Based Syst..

[53]  Kan Li,et al.  A deeper graph neural network for recommender systems , 2019, Knowl. Based Syst..

[54]  M. M. Malik,et al.  Data mining and predictive analytics applications for the delivery of healthcare services: a systematic literature review , 2016, Annals of Operations Research.

[55]  Tomohiro Ando,et al.  Merchant selection and pricing strategy for a platform firm in the online group buying market , 2018, Ann. Oper. Res..

[56]  M. de Rijke,et al.  Personalised Reranking of Paper Recommendations Using Paper Content and User Behavior , 2019, ACM Trans. Inf. Syst..

[57]  M. de Rijke,et al.  Joint Neural Collaborative Filtering for Recommender Systems , 2019, ACM Trans. Inf. Syst..

[58]  J. Suykens,et al.  Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research , 2015, Eur. J. Oper. Res..

[59]  Gilles Louppe,et al.  Understanding variable importances in forests of randomized trees , 2013, NIPS.

[60]  Amir H. Gandomi,et al.  Resolving data sparsity and cold start problem in collaborative filtering recommender system using Linked Open Data , 2020, Expert Syst. Appl..

[61]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[62]  Michael A. Shepherd,et al.  Effect of task on time spent reading as an implicit measure of interest , 2005, ASIST.

[63]  Konstantinos G. Margaritis,et al.  Using SVD and demographic data for the enhancement of generalized Collaborative Filtering , 2007, Inf. Sci..

[64]  Chenglin Li,et al.  Android Malware Detection Based on Factorization Machine , 2018, IEEE Access.

[65]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[66]  Stefan Lessmann,et al.  Can Deep Learning Predict Risky Retail Investors? A Case Study in Financial Risk Behavior Forecasting , 2018, Eur. J. Oper. Res..

[67]  Bart Baesens,et al.  A modified Pareto/NBD approach for predicting customer lifetime value , 2007, Expert Syst. Appl..