Discriminative conditional restricted Boltzmann machine for discrete choice and latent variable modelling

Conventional methods of estimating latent behaviour generally use attitudinal questions which are subjective and these survey questions may not always be available. We hypothesize that an alternative approach can be used for latent variable estimation through an undirected graphical models. For instance, non-parametric artificial neural networks. In this study, we explore the use of generative non-parametric modelling methods to estimate latent variables from prior choice distribution without the conventional use of measurement indicators. A restricted Boltzmann machine is used to represent latent behaviour factors by analyzing the relationship information between the observed choices and explanatory variables. The algorithm is adapted for latent behaviour analysis in discrete choice scenario and we use a graphical approach to evaluate and understand the semantic meaning from estimated parameter vector values. We illustrate our methodology on a financial instrument choice dataset and perform statistical analysis on parameter sensitivity and stability. Our findings show that through non-parametric statistical tests, we can extract useful latent information on the behaviour of latent constructs through machine learning methods and present strong and significant influence on the choice process. Furthermore, our modelling framework shows robustness in input variability through sampling and validation.

[1]  Bilal Farooq,et al.  Next Direction Route Choice Model for Cyclist Using Panel Data , 2016 .

[2]  Bilal Farooq,et al.  Pedestrian Activity Pattern Mining in WiFi-Network Connection Data , 2016 .

[3]  B. Schölkopf,et al.  Modeling Human Motion Using Binary Latent Variables , 2007 .

[4]  Michel Bierlaire,et al.  Forecasting the Demand for Electric Vehicles: Accounting for Attitudes and Perceptions , 2014, Transp. Sci..

[5]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  A. Daly,et al.  Handbook of Choice Modelling , 2014 .

[7]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[8]  Joan L. Walker,et al.  Hybrid Choice Models: Progress and Challenges , 2002 .

[9]  Ali Yazdizadeh,et al.  A Generic Form for Capturing Unobserved Heterogeneity in Discrete Choice Modelling: Application to Neighborhood Location Choice , 2016 .

[10]  Chandra R. Bhat,et al.  A New Estimation Approach to Integrate Latent Psychological Constructs in Choice Modeling , 2014 .

[11]  Jordan J. Louviere,et al.  Latent variables in discrete choice experiments , 2012 .

[12]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[13]  Nicolas Le Roux,et al.  Representational Power of Restricted Boltzmann Machines and Deep Belief Networks , 2008, Neural Computation.

[14]  H. Timmermans,et al.  Expanding scope of hybrid choice models allowing for mixture of social influences and latent attitudes: Application to intended purchase of electric cars , 2014 .

[15]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[16]  Manish Aggarwal,et al.  On Learning of Choice Models with Interactive Attributes , 2016, IEEE Transactions on Knowledge and Data Engineering.

[17]  Dina L. Denham,et al.  Hinton diagrams: Viewing connection strengths in neural networks , 1994 .

[18]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[19]  M. Bierlaire,et al.  Discrete Choice Methods and their Applications to Short Term Travel Decisions , 1999 .

[20]  Takayuki Osogami,et al.  A Deep Choice Model , 2016, AAAI.

[21]  Michael I. Jordan,et al.  On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes , 2001, NIPS.

[22]  Reinhard Madlener,et al.  Consumer Preferences for Alternative Fuel Vehicles: A Discrete Choice Analysis , 2012 .

[23]  Miguel Á. Carreira-Perpiñán,et al.  On Contrastive Divergence Learning , 2005, AISTATS.

[24]  Amir F. Atiya,et al.  An Empirical Comparison of Machine Learning Models for Time Series Forecasting , 2010 .

[25]  D. Donoho.  50 Years of Data Science , 2017 .

[26]  D. Hamby.  A review of techniques for parameter sensitivity analysis of environmental models , 1994, Environmental monitoring and assessment.

[27]  J. Polak,et al.  Exploring the role of individual attitudes and perceptions in predicting the demand for cycling: a hybrid choice modelling approach , 2014 .

[28]  David A. Hensher,et al.  The Mixed Logit Model: the State of Practice and Warnings for the Unwary , 2001 .

[29]  Jon C. Helton,et al.  Sensitivity analysis techniques and results for performance assessment at the Waste Isolation Pilot Plant , 1991 .

[30]  J. Vermunt,et al.  Latent class cluster analysis , 2002 .

[31]  M. Bierlaire,et al.  Attitudes towards mode choice in Switzerland , 2013 .

[32]  Caspar G. Chorus,et al.  On the (im-)possibility of deriving transport policy implications from hybrid choice models , 2014 .

[33]  Jennifer A. Thacher,et al.  Using Angler Characteristics and Attitudinal Data to Identify Environmental Preference Classes: A Latent-Class Model , 2006 .

[34]  Yoshua Bengio,et al.  Classification using discriminative restricted Boltzmann machines , 2008, ICML '08.

[35]  Kalidas Ashok,et al.  Extending Discrete Choice Models to Incorporate Attitudinal and Other Latent Variables , 2002 .

[36]  Jeremy Shires,et al.  Accommodating Underlying Pro-environmental Attitudes in a Rail Travel Context: Application of a Latent Variable Latent Class Specification , 2013 .

[37]  Paola Annoni,et al.  Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index , 2010, Comput. Phys. Commun..

[38]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[39]  Leo Breiman,et al.  Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) , 2001, Statistical Science.

[40]  Nando de Freitas,et al.  Active Preference Learning with Discrete Choice Data , 2007, NIPS.

[41]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[42]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[43]  Leo Breiman,et al.  Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) , 2001 .

[44]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[45]  Joan L. Walker,et al.  How, when and why integrated choice and latent variable models are latently useful , 2016 .

[46]  Amos Azaria,et al.  Combining psychological models with machine learning to better predict people’s decisions , 2012, Synthese.

[47]  Takayuki Osogami,et al.  Restricted Boltzmann machines modeling human choice , 2014, NIPS.

[48]  Geoffrey E. Hinton,et al.  Conditional Restricted Boltzmann Machines for Structured Output Prediction , 2011, UAI.