Forecasting of Commercial Sales with Large Scale Gaussian Processes

This paper argues that there has not been enough discussion in the field of applications of Gaussian Process for the fast moving consumer goods industry. Yet, this technique can be important as it e.g., can provide automatic feature relevance determination and the posterior mean can unlock insights on the data. Significant challenges are the large size and high dimensionality of commercial data at a point of sale. The study reviews approaches in the Gaussian Processes modeling for large data sets, evaluates their performance on commercial sales and shows value of this type of models as a decision-making tool for management.

[1]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[2]  Neil D. Lawrence,et al.  Parallelizable sparse inverse formulation Gaussian processes (SpInGP) , 2016, 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP).

[3]  Alexis Boukouvalas,et al.  GPflow: A Gaussian Process Library using TensorFlow , 2016, J. Mach. Learn. Res..

[4]  Evgeny Burnaev,et al.  Minimax Approach to Variable Fidelity Data Interpolation , 2017, AISTATS.

[5]  Dustin Tran,et al.  Edward: A library for probabilistic modeling, inference, and criticism , 2016, ArXiv.

[6]  Xiaofeng Meng,et al.  Short-Term Wind Power Forecasting Using Gaussian Processes , 2013, IJCAI.

[7]  Asim Ansari,et al.  Bayesian Nonparametric Customer Base Analysis with Model-based Visualizations , 2016, Mark. Sci..

[8]  Girma Kejela.  Short-term forecasting of electricity consumption using Gaussian processes , 2012 .

[9]  Tao Chen,et al.  Bagging for Gaussian process regression , 2009, Neurocomputing.

[10]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[11]  Douglas W. Nychka,et al.  Covariance Tapering for Likelihood-Based Estimation in Large Spatial Data Sets , 2008 .

[12]  Maziar Raissi,et al.  Parametric Gaussian process regression for big data , 2017, Computational Mechanics.

[13]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[14]  Kian Hsiang Low,et al.  A Distributed Variational Inference Framework for Unifying Parallel Sparse Gaussian Process Regression Models , 2016, ICML.

[15]  D. Nychka,et al.  Covariance Tapering for Interpolation of Large Spatial Datasets , 2006 .

[16]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[17]  Carl E. Rasmussen,et al.  Analysis of Some Methods for Reduced Rank Gaussian Process Regression , 2003, European Summer School on Multi-AgentControl.

[18]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[19]  Ye Wang,et al.  Gaussian-Process-Based Demand Forecasting for Predictive Control of Drinking Water Networks , 2014, CRITIS.

[20]  Alexey Zaytsev,et al.  Surrogate modeling of multifidelity data for large samples , 2015 .

[21]  Chiwoo Park,et al.  Patchwork Kriging for Large-scale Gaussian Process Regression , 2017, J. Mach. Learn. Res..

[22]  Robert B. Gramacy,et al.  Massively parallel approximate Gaussian process regression , 2013, SIAM/ASA J. Uncertain. Quantification.

[23]  T. Gneiting.  Compactly Supported Correlation Functions , 2002 .

[24]  Neil D. Lawrence,et al.  Introduction to Gaussian Processes , 2013 .

[25]  Aki Vehtari,et al.  Modelling local and global phenomena with sparse Gaussian processes , 2008, UAI.

[26]  Volker Tresp,et al.  A Bayesian Committee Machine , 2000, Neural Computation.

[27]  Maxim Panov,et al.  Regression on the basis of nonstationary Gaussian processes with Bayesian regularization , 2016 .

[28]  Kian Hsiang Low,et al.  A Generalized Stochastic Variational Bayesian Hyperparameter Learning Framework for Sparse Spectrum Gaussian Process Regression , 2016, AAAI.

[29]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[30]  Wenjie Huang,et al.  A Novel Trigger Model for Sales Prediction with Data Mining Techniques , 2015, Data Sci. J..

[31]  Leslie Greengard,et al.  Fast Direct Methods for Gaussian Processes and the Analysis of NASA Kepler Mission Data , 2014 .

[32]  A. Gelfand,et al.  Gaussian predictive process models for large spatial data sets , 2008, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[33]  J. Weston,et al.  Approximation Methods for Gaussian Process Regression , 2007 .

[34]  Kian Hsiang Low,et al.  A Unifying Framework of Anytime Sparse Gaussian Process Regression Models with Stochastic Variational Inference for Big Data , 2015, ICML.

[35]  Limin Sun,et al.  Prediction of Tobacco Sales Based on Support Vector Machine , 2015 .

[36]  John Salvatier,et al.  Probabilistic programming in Python using PyMC3 , 2016, PeerJ Comput. Sci..

[37]  Evgeny Burnaev,et al.  Adaptive Design of Experiments Based on Gaussian Processes , 2015, SLDS.

[38]  Asim Ansari,et al.  Model-based Dashboards for Customer Analytics , 2015 .

[39]  E. Monte,et al.  Regional Tourism Demand Forecasting with Machine Learning Models: Gaussian Process Regression vs. Neural Network Models in a Multiple-Input Multiple-Output Setting , 2017 .

[40]  Kian Hsiang Low,et al.  Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations , 2013, UAI.

[41]  Carl E. Rasmussen,et al.  Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models , 2014, NIPS.

[42]  Zoubin Ghahramani,et al.  The Random Forest Kernel and creating other kernels for big data from random partitions , 2014 .

[43]  Scott Shenker,et al.  Spark: Cluster Computing with Working Sets , 2010, HotCloud.

[44]  H. Rue,et al.  An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach , 2011 .

[45]  Daniel Foreman-Mackey,et al.  Fast and Scalable Gaussian Process Modeling with Applications to Astronomical Time Series , 2017, 1703.09710.

[46]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[47]  Jan Peters,et al.  Model Learning with Local Gaussian Process Regression , 2009, Adv. Robotics.

[48]  Martin A. Riedmiller,et al.  Electricity Demand Forecasting using Gaussian Processes , 2013, AAAI Workshop: Trading Agent Design and Analysis.

[49]  Yu Ding,et al.  Domain Decomposition Approach for Fast Gaussian Process Regression of Large Spatial Data Sets , 2011, J. Mach. Learn. Res..

[50]  Sourish Das,et al.  Fast Gaussian Process Regression for Big Data , 2015, Big Data Res..

[51]  Marc Peter Deisenroth,et al.  Distributed Gaussian Processes , 2015, ICML.

[52]  Evgeny Burnaev,et al.  Large scale variable fidelity surrogate modeling , 2017, Annals of Mathematics and Artificial Intelligence.

[53]  Robert B. Gramacy,et al.  Ja n 20 08 Bayesian Treed Gaussian Process Models with an Application to Computer Modeling , 2009 .

[54]  Chiwoo Park,et al.  Efficient Computation of Gaussian Process Regression for Large Spatial Data Sets by Patching Local Gaussian Processes , 2016, J. Mach. Learn. Res..

[55]  Evgeny Burnaev,et al.  Computationally efficient algorithm for Gaussian Process regression in case of structured samples , 2016, Computational Mathematics and Mathematical Physics.

[56]  Kian Hsiang Low,et al.  Parallel Gaussian Process Regression for Big Data: Low-Rank Representation Meets Markov Approximation , 2014, AAAI.

[57]  Neil D. Lawrence,et al.  Fast Forward Selection to Speed Up Sparse Gaussian Process Regression , 2003, AISTATS.

[58]  Michael Minyi Zhang,et al.  Embarrassingly Parallel Inference for Gaussian Processes , 2017, J. Mach. Learn. Res..

[59]  Trevor Darrell,et al.  Sparse probabilistic regression for activity-independent human pose inference , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[60]  Michael A. Osborne,et al.  Gaussian process regression for forecasting battery state of health , 2017, 1703.05687.

[61]  Elad Gilboa,et al.  Scaling Multidimensional Gaussian Processes using Projected Additive Approximations , 2013, ICML.

[62]  Daniel W. Apley,et al.  Local Gaussian Process Approximation for Large Computer Experiments , 2013, 1303.0383.

[63]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[64]  Rob Law,et al.  A sparse Gaussian process regression model for tourism demand forecasting in Hong Kong , 2012, Expert Syst. Appl..

[65]  Tian-Shyug Lee,et al.  Sales forecasting for computer wholesalers: A comparison of multivariate adaptive regression splines and artificial neural networks , 2012, Decis. Support Syst..

[66]  Kristian Kersting,et al.  pyGPs: a Python library for Gaussian process regression and classification , 2015, J. Mach. Learn. Res..

[67]  Evgeny Burnaev,et al.  GTApprox: Surrogate modeling for industrial design , 2016, Adv. Eng. Softw..

[68]  Neil D. Lawrence,et al.  Gaussian Processes for Big Data , 2013, UAI.

[69]  Evgeny V. Burnaev,et al.  Properties of the posterior distribution of a regression model based on Gaussian random fields , 2013, Autom. Remote. Control..

[70]  Zoubin Ghahramani,et al.  Local and global sparse Gaussian process approximations , 2007, AISTATS.

[71]  Evgeny Burnaev,et al.  Gaussian Process Regression for Structured Data Sets , 2015, SLDS.

[72]  Arno Solin,et al.  Variational Fourier Features for Gaussian Processes , 2016, J. Mach. Learn. Res..

[73]  Carl E. Rasmussen,et al.  Infinite Mixtures of Gaussian Process Experts , 2001, NIPS.

[74]  Jian Tan.  Guizhou Cigarette Sales Prediction based on Seasonal Decomposition MLP , 2015 .

[75]  Bruce E. Ankenman,et al.  Comparison of Gaussian process modeling software , 2016, 2016 Winter Simulation Conference (WSC).

[76]  Charles W. Chase,et al.  Demand-Driven Forecasting: A Structured Approach to Forecasting , 2009 .

[77]  Lehel Csató,et al.  Sparse On-Line Gaussian Processes , 2002, Neural Computation.