Fitted Q-iteration and functional networks for ubiquitous recommender systems

Ubiquitous recommender systems facilitate users on-location by personalized recommendations of items in the proximity via mobile devices. Due to a high variability of situations and preferences, an efficient resource processing is needed in order to assist the user in a proper way. In this paper, we consider a recommender system, able to learn preferences/habits of users through contextual information, such as location and time, using a new offline model-free approximate Q-iteration. Following the basic idea of Fitted Q-Iteration, the paper focuses on a computational scheme, based on functional networks, and that, unlike the well-known neural ones, does not require a large number of training samples. A preliminary case study, which deals with a shopping mall, is useful to show the approximation capabilities of the proposed approach.

[1]  Enrique F. Castillo,et al.  A Minimax Method for Learning Functional Networks , 2000, Neural Processing Letters.

[2]  Matteo Gaeta,et al.  An extended functional network model and its application for a gas sensing system , 2013, Soft Comput..

[3]  Emad A. El-Sebakhy.  New Computational Intelligence Paradigm for Estimating the Software Project Effort , 2008, 22nd International Conference on Advanced Information Networking and Applications - Workshops (aina workshops 2008).

[4]  Yong-Quan Zhou,et al.  Application of Functional Network to Solving Classification Problems , 2005, IEC.

[5]  Paul Resnick,et al.  Recommender systems , 1997, CACM.

[6]  Cristina Solares,et al.  A First Approach to Solve Classification Problems Based on Functional Networks , 2005, ICANN.

[7]  Ahmad A. Kardan,et al.  A hybrid web recommender system based on Q-learning , 2008, SAC '08.

[8]  Pierre Geurts,et al.  Tree-Based Batch Mode Reinforcement Learning , 2005, J. Mach. Learn. Res..

[9]  Stefania Tomasiello,et al.  A functional network to predict fresh and hardened properties of self‐compacting concretes , 2011 .

[10]  Ali S. Hadi,et al.  Iterative Least Squares Functional Networks Classifier , 2007, IEEE Transactions on Neural Networks.

[11]  Amparo Alonso-Betanzos,et al.  Feature Selection Based on Sensitivity Analysis , 2007, CAEPIA.

[12]  Carlos Dafonte,et al.  A comparison between functional networks and artificial neural networks for the prediction of fishing catches , 2004, Neural Computing & Applications.

[13]  Matteo Gaeta,et al.  Fitted Q-iteration by Functional Networks for control problems , 2016 .

[14]  Jan Peters,et al.  Fitted Q-iteration by Advantage Weighted Regression , 2008, NIPS.

[15]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[16]  Csaba Szepesvári,et al.  Fitted Q-iteration in continuous action-space MDPs , 2007, NIPS.

[17]  Gediminas Adomavicius,et al.  Context-aware recommender systems , 2008, RecSys '08.

[18]  S. Timmer,et al.  Fitted Q Iteration with CMACs , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[19]  Amparo Alonso-Betanzos,et al.  Classification of computer intrusions using functional networks. A comparative study , 2007, ESANN.

[20]  Stephen Shaoyi Liao,et al.  A novel mobile recommender system for indoor shopping , 2012, Expert Syst. Appl..

[21]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[22]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[23]  Martin A. Riedmiller.  Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method , 2005, ECML.

[24]  S. Singh,et al.  Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System , 2011, J. Artif. Intell. Res..

[25]  Matteo Gaeta,et al.  A Generalized Functional Network for a Classifier‐Quantifiers Scheme in a Gas‐Sensing System , 2013, Int. J. Intell. Syst..

[26]  Michael Bruen,et al.  Functional networks in real-time flood forecasting—a novel application , 2005 .

[27]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[28]  Tariq Mahmood,et al.  Dynamic personalization in conversational recommender systems , 2014, Inf. Syst. E Bus. Manag..

[29]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[30]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[31]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[32]  Vasantha Kalyani David,et al.  Pattern Recognition using Neural and Functional Networks , 2009, Studies in Computational Intelligence.

[33]  Tarek Helmy,et al.  Hybrid Computational Intelligence Models for Porosity and Permeability Prediction of Petroleum reservoirs , 2010, Int. J. Comput. Intell. Appl..

[34]  Tariq Mahmood,et al.  Comparing reward-based optimal behaviors in user-adapted Recommender Systems , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[35]  Michael Frankfurter,et al.  Numerical Recipes In C The Art Of Scientific Computing , 2016 .

[36]  Gregory D. Abowd,et al.  A Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-Aware Applications , 2001, Hum. Comput. Interact..

[37]  Enrique F. Castillo,et al.  Functional networks with applications: a neural-based paradigm [Book Review] , 1999, IEEE Transactions on Neural Networks.

[38]  Linas Baltrunas,et al.  Exploiting contextual information in recommender systems , 2008, RecSys '08.

[39]  Beatriz Lacruz,et al.  International Conference on Mathematical and Statistical Modeling in Honor of Enrique Castillo . June 28-30 , 2006 Functional Networks for Classification and Regression Problems , 2006 .

[40]  Andrew E. Fano,et al.  Shopper's eye: using location-based filtering for a shopping agent in the physical world , 1998, AGENTS '98.

[41]  E. Castillo,et al.  Working with differential, functional and difference equations using functional networks , 1999 .

[42]  Dirk Ormoneit,et al.  Kernel-Based Reinforcement Learning , 2017, Encyclopedia of Machine Learning and Data Mining.

[43]  Geoffrey J. Gordon.  Online Fitted Reinforcement Learning , 1995 .

[44]  George Angelos Papadopoulos,et al.  Ubiquitous recommender systems , 2013, Computing.

[45]  K. Thiruvenkatasamy,et al.  Tidal level forecasting using functional and sequential learning neural networks , 2006 .