How to effectively use machine learning models to predict the solutions for optimization problems: lessons from loss function

Using machine learning in solving constraint optimization and combinatorial problems is becoming an active research area in both computer science and operations research communities. This paper aims to predict a good solution for constraint optimization problems using advanced machine learning techniques. It extends the work of \cite{abbasi2020predicting} to use machine learning models for predicting the solution of large-scaled stochastic optimization models by examining more advanced algorithms and various costs associated with the predicted values of decision variables. It also investigates the importance of loss function and error criterion in machine learning models where they are used for predicting solutions of optimization problems. We use a blood transshipment problem as the case study. The results for the case study show that LightGBM provides promising solutions and outperforms other machine learning models used by \cite{abbasi2020predicting} specially when mean absolute deviation criterion is used.

[1]  J. Boylan,et al.  On the stock control performance of intermittent demand estimators , 2006 .

[2]  Marco E. Lübbecke,et al.  Learning When to Use a Decomposition , 2017, CPAIOR.

[3]  Richard Gerlach,et al.  Demand forecasting in supply chain: The impact of demand volatility in the presence of promotion , 2019, Comput. Ind. Eng..

[4]  Casper Solheim Bojer,et al.  Kaggle forecasting competitions: An overlooked learning opportunity , 2020, ArXiv.

[5]  M. Zamani,et al.  Considering pricing and uncertainty in designing a reverse logistics network , 2019, International Journal of Industrial and Systems Engineering.

[6]  J. Scott Armstrong,et al.  On the Selection of Error Measures for Comparisons Among Forecasting Methods , 2005 .

[7]  Babak Abbasi,et al.  An age-based lateral-transshipment policy for perishable items , 2018 .

[8]  Michèle Hibon,et al.  Exponential smoothing: The effect of initial values and loss functions on post-sample forecasting accuracy , 1991 .

[9]  Rob J Hyndman,et al.  Machine learning applications in time series hierarchical forecasting , 2019, ArXiv.

[10]  Kate Smith-Miles,et al.  Cross-disciplinary perspectives on meta-learning for algorithm selection , 2009, CSUR.

[11]  Aris A. Syntetos,et al.  Accuracy and Accuracy Implication Metrics for Intermittent Demand , 2006 .

[12]  Yoshua Bengio,et al.  Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon , 2018, Eur. J. Oper. Res..

[13]  F. Diebold,et al.  Optimal Prediction Under Asymmetric Loss , 1994, Econometric Theory.

[14]  A. Syntetos,et al.  Forecast errors and inventory performance under forecast information sharing , 2012 .

[15]  Babak Abbasi,et al.  Proactive transshipment in the blood supply chain: A stochastic programming approach , 2019 .

[16]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Babak Abbasi,et al.  Predicting solutions of large-scale optimization problems via machine learning: A case study in blood supply chain management , 2020, Comput. Oper. Res..

[18]  Louis Wehenkel,et al.  Online Learning for Strong Branching Approximation in Branch-and-Bound , 2016 .

[19]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[20]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[21]  P. Groenen,et al.  Nonlinear Forecasting with Many Predictors Using Kernel Ridge Regression , 2011 .

[22]  Yoshua Bengio,et al.  Predicting Tactical Solutions to Operational Planning Problems under Imperfect Information , 2018, INFORMS J. Comput..

[23]  Fotios Petropoulos,et al.  An evaluation of simple versus complex selection rules for forecasting many time series , 2014 .

[24]  Lisa Werner,et al.  Principles of forecasting: A handbook for researchers and practitioners , 2002 .

[25]  Fotios Petropoulos,et al.  'Horses for Courses' in demand forecasting , 2014, Eur. J. Oper. Res..

[26]  M. Kosorok Introduction to Empirical Processes and Semiparametric Inference , 2008 .

[27]  D. Basak,et al.  Support Vector Regression , 2008 .

[28]  Ali Eshragh,et al.  Demand forecasting in the presence of systematic events: Cases in capturing sales promotions , 2019, International Journal of Production Economics.

[29]  Jonathan T. Barron,et al.  A General and Adaptive Robust Loss Function , 2017, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[31]  Valentin Flunkert,et al.  DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks , 2017, International Journal of Forecasting.

[32]  L. Papageorgiou,et al.  Customer Demand Forecasting via Support Vector Regression Analysis , 2005 .

[33]  Michael P. Clements,et al.  On the limitations of comparing mean square forecast errors , 1993 .

[34]  Premysl Sucha,et al.  Accelerating the Branch-and-Price Algorithm Using Machine Learning , 2018, Eur. J. Oper. Res..

[35]  Rob J. Hyndman,et al.  FFORMA: Feature-based forecast model averaging , 2020, International Journal of Forecasting.

[36]  Daniel Delahaye,et al.  Multi-label Classification for the Generation of Sub-problems in Time-constrained Combinatorial Optimization , 2019, ICORES.

[37]  Konstantinos Nikolopoulos,et al.  Supply chain forecasting: Theory, practice, their gap and the future , 2016, Eur. J. Oper. Res..

[38]  Andrea Lodi,et al.  On learning and branching: a survey , 2017 .

[39]  George B. Dantzig,et al.  Linear Programming Under Uncertainty , 2004, Manag. Sci..