Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities

Decision-focused learning (DFL) is an emerging paradigm in machine learning which trains a model to optimize decisions, integrating prediction and optimization in an end-to-end system. This paradigm holds the promise to revolutionize decision-making in many real-world applications which operate under uncertainty, where the estimation of unknown parameters within these decision models often becomes a substantial roadblock. This paper presents a comprehensive review of DFL. It provides an in-depth analysis of the various techniques devised to integrate machine learning and optimization models introduces a taxonomy of DFL methods distinguished by their unique characteristics, and conducts an extensive empirical evaluation of these methods proposing suitable benchmark dataset and tasks for DFL. Finally, the study provides valuable insights into current and potential future avenues in DFL research.

[1]  A smart predict-then-optimize method for targeted and cost-effective maritime transportation , 2023, Transportation Research Part B: Methodological.

[2]  Paul Grigas,et al.  Active Learning in the Predict-then-Optimize Framework: A Margin-Based Approach , 2023, ArXiv.

[3]  Jasper C. H. Lee,et al.  Branch & Learn with Post-hoc Correction for Predict+Optimize with Unknown Parameters in Constraints , 2023, CPAIOR.

[4]  Elias Boutros Khalil,et al.  Multi-Task Predict-then-Optimize , 2022, 2212.05403.

[5]  Ying Yang,et al.  Pairwise-Comparison Based Semi-SPO Method for Ship Inspection Planning in Maritime Transportation , 2022, Journal of Marine Science and Engineering.

[6]  Benoit Steiner,et al.  SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problems , 2022, ICML.

[7]  Mausam,et al.  A Solver-Free Framework for Scalable Learning in Neural ILP Architectures , 2022, NeurIPS.

[8]  M. Winkenbach,et al.  2021 Amazon Last Mile Routing Research Challenge: Data Set , 2022, Transportation Science.

[9]  Jasper C. H. Lee,et al.  Predict+Optimize for Packing and Covering LPs with Unknown Parameters in Constraints , 2022, AAAI.

[10]  Axel Parmentier,et al.  Learning with Combinatorial Optimization Layers: a Probabilistic Approach , 2022, ArXiv.

[11]  Yinliang Xu,et al.  Electricity Price Prediction for Energy Storage System Arbitrage: A Decision-Focused Approach , 2022, IEEE Transactions on Smart Grid.

[12]  Elias Boutros Khalil,et al.  PyEPO: A PyTorch-based End-to-End Predict-then-Optimize Library for Linear and Integer Programming , 2022, ArXiv.

[13]  Ali Ugur Guler,et al.  A Divide and Conquer Algorithm for Predict+Optimize with Non-convex Problems , 2022, AAAI.

[14]  Subham S. Sahoo,et al.  Backpropagation through Combinatorial Algorithms: Identity with Projection Works , 2022, ICLR.

[15]  Jasper C. H. Lee,et al.  Branch & Learn for Recursively and Iteratively Solvable Problems in Predict+Optimize , 2022, NeurIPS.

[16]  Kai-Kit Wong,et al.  Port Selection for Fluid Antenna Systems , 2022, IEEE Communications Letters.

[17]  Tias Guns,et al.  Decision-Focused Learning: Through the Lens of Learning to Rank , 2021, ICML.

[18]  Pascal Van Hentenryck,et al.  End-to-End Learning for Fair Ranking Systems , 2021, WWW.

[19]  Paul Grigas,et al.  Risk Bounds and Calibration for a Smart Predict-then-Optimize Method , 2021, NeurIPS.

[20]  Tias Guns,et al.  Data Driven VRP: A Neural Network Model to Learn Hidden Preferences for VRP , 2021, ArXiv.

[21]  Kevin Leyton-Brown,et al.  The Perils of Learning Before Optimizing , 2021, AAAI.

[22]  Pasquale Minervini,et al.  Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions , 2021, NeurIPS.

[23]  G. Martius,et al.  CombOptNet: Fit the Right NP-Hard Problem by Learning Integer Programming Constraints , 2021, ICML.

[24]  Jakob Uszkoreit,et al.  Differentiable Patch Selection for Image Recognition , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Hongrui Chu,et al.  Data-driven optimization for last-mile delivery , 2021, Complex & Intelligent Systems.

[26]  Michelangelo Diligenti,et al.  Contrastive Losses and Solution Caching for Predict-and-Optimize , 2020, IJCAI.

[27]  Kai-Kit Wong,et al.  Fluid Antenna System for 6G: When Bruce Lee Inspires Wireless Communications , 2020, Electronics Letters.

[28]  Tias Guns,et al.  Interior Point Solving for LP-based prediction+optimisation , 2020, NeurIPS.

[29]  Daria Terekhov,et al.  Learning Linear Programs from Optimal Decisions , 2020, NeurIPS.

[30]  Daniel Tarlow,et al.  Gradient Estimation with Stochastic Softmax Tricks , 2020, NeurIPS.

[31]  Kotagiri Ramamohanarao,et al.  Dynamic Programming for Predict+Optimise , 2020, AAAI.

[32]  Georg Martius,et al.  Deep Graph Matching via Blackbox Differentiation of Combinatorial Solvers , 2020, ECCV.

[33]  Adam N. Elmachtoub,et al.  Decision Trees for Decision-Making under the Predict-then-Optimize Framework , 2020, ICML.

[34]  Francis Bach,et al.  Learning with Differentiable Perturbed Optimizers , 2020, ArXiv.

[35]  Mathieu Blondel,et al.  Fast Differentiable Sorting and Ranking , 2020, ICML.

[36]  Yonina C. Eldar,et al.  Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing , 2019, IEEE Signal Processing Magazine.

[37]  G. Martius,et al.  Optimizing Rank-Based Metrics With Blackbox Differentiation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  G. Martius,et al.  Differentiation of Blackbox Combinatorial Solvers , 2019, ICLR.

[39]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[40]  Stephen P. Boyd,et al.  Differentiable Convex Optimization Layers , 2019, NeurIPS.

[41]  Jason Baldridge,et al.  Learning Dense Representations for Entity Retrieval , 2019, CoNLL.

[42]  Milind Tambe,et al.  MIPaaL: Mixed Integer Program as a Layer , 2019, AAAI.

[43]  Vladlen Koltun,et al.  The Limited Multi-Label Projection Layer , 2019, ArXiv.

[44]  James Bailey,et al.  An Investigation into Prediction + Optimisation for the Knapsack Problem , 2019, CPAIOR.

[45]  Milind Tambe,et al.  End to end learning and optimization on graphs , 2019, NeurIPS.

[46]  Priya L. Donti,et al.  SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver , 2019, ICML.

[47]  Ambuj Tewari,et al.  Generalization Bounds in the Predict-then-Optimize Framework , 2019, NeurIPS.

[48]  Stephen P. Boyd,et al.  Differentiating through a cone program , 2019, Journal of Applied and Numerical Optimization.

[49]  Daria Terekhov,et al.  Deep Inverse Optimization , 2018, CPAIOR.

[50]  Stephen P. Boyd,et al.  Solution refinement at regular points of conic problems , 2018, Computational Optimization and Applications.

[51]  Milind Tambe,et al.  Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization , 2018, AAAI.

[52]  Abien Fred Agarap Deep Learning using Rectified Linear Units (ReLU) , 2018, ArXiv.

[53]  Hoay Beng Gooi,et al.  Toward Optimal Energy Management of Microgrids via Robust Two-Stage Optimization , 2018, IEEE Transactions on Smart Grid.

[54]  Thorsten Joachims,et al.  Fairness of Exposure in Rankings , 2018, KDD.

[55]  Adam N. Elmachtoub,et al.  Smart "Predict, then Optimize" , 2017, Manag. Sci..

[56]  J. Zico Kolter,et al.  OptNet: Differentiable Optimization as a Layer in Neural Networks , 2017, ICML.

[57]  Priya L. Donti,et al.  Task-based End-to-end Model Learning in Stochastic Optimization , 2017, NIPS.

[58]  Anoop Cherian,et al.  On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization , 2016, ArXiv.

[59]  Ramón Fernández Astudillo,et al.  From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification , 2016, ICML.

[60]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  Barak A. Pearlmutter,et al.  Automatic differentiation in machine learning: a survey , 2015, J. Mach. Learn. Res..

[62]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[63]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[64]  P. Toth,et al.  Vehicle Routing: Problems, Methods, and Applications, Second Edition , 2014 .

[65]  Ian J. Goodfellow,et al.  On distinguishability criteria for estimating generative models , 2014, ICLR.

[66]  Eugene W. Myers,et al.  Active Graph Matching for Automatic Joint Segmentation and Annotation of C. elegans , 2014, MICCAI.

[67]  Dimitris Bertsimas,et al.  From Predictive to Prescriptive Analytics , 2014, Manag. Sci..

[68]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[69]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[70]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[71]  Barry O'Sullivan,et al.  Properties of Energy-Price Forecasts for Scheduling , 2012, CP.

[72]  Lia Purpura On Tools , 2012 .

[73]  Yee Whye Teh,et al.  A fast and simple algorithm for training neural probabilistic language models , 2012, ICML.

[74]  Justin Domke,et al.  Generic Methods for Optimization-Based Modeling , 2012, AISTATS.

[75]  George Papandreou,et al.  Perturb-and-MAP random fields: Using discrete optimization to learn and sample from energy models , 2011, 2011 International Conference on Computer Vision.

[76]  Sebastian Nowozin,et al.  Structured Learning and Prediction in Computer Vision , 2011, Found. Trends Comput. Graph. Vis..

[77]  Sanyogita Lakhera,et al.  Approximating electrical distribution networks via mixed-integer nonlinear programming , 2011 .

[78]  Justin Domke,et al.  Implicit Differentiation by Perturbation , 2010, NIPS.

[79]  Yinyu Ye,et al.  Distributionally Robust Optimization Under Moment Uncertainty with Application to Data-Driven Problems , 2010, Oper. Res..

[80]  Aapo Hyvärinen,et al.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models , 2010, AISTATS.

[81]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[82]  H. Zou,et al.  NEW MULTICATEGORY BOOSTING ALGORITHMS BASED ON MULTICATEGORY FISHER-CONSISTENT LOSSES. , 2008, The annals of applied statistics.

[83]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[84]  Tie-Yan Liu,et al.  Learning to rank: from pairwise approach to listwise approach , 2007, ICML '07.

[85]  P. Massart,et al.  Risk bounds for statistical learning , 2007, math/0702683.

[86]  David Pisinger,et al.  Where are the hard knapsack problems? , 2005, Comput. Oper. Res..

[87]  Chelsea C. White,et al.  Optimal vehicle routing with real-time traffic information , 2005, IEEE Transactions on Intelligent Transportation Systems.

[88]  Nikolaos V. Sahinidis,et al.  Optimization under uncertainty: state-of-the-art and opportunities , 2004, Comput. Chem. Eng..

[89]  Lorenzo Garlappi,et al.  Portfolio Selection with Parameter and Model Uncertainty: A Multi-Prior Approach , 2004 .

[90]  Baoding Liu,et al.  Theory and Practice of Uncertain Programming , 2003, Studies in Fuzziness and Soft Computing.

[91]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[92]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[93]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[94]  O. Barndorff-Nielsen Information and Exponential Families in Statistical Theory , 1980 .

[95]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[96]  Meredith L. Gore,et al.  Predicting Wildlife Trafficking Routes with Differentiable Shortest Paths , 2023, CPAIOR.

[97]  Adam N. Elmachtoub,et al.  Estimate-Then-Optimize Versus Integrated-Estimation-Optimization: A Stochastic Dominance Perspective , 2023, ArXiv.

[98]  Sonja Johnson-Yu,et al.  Modeling Robustness in Decision-Focused Learning as a Stackelberg Game , 2023, AAMAS.

[99]  More than accuracy: end-to-end wind power forecasting that optimises the energy system , 2023, Electric Power Systems Research.

[100]  Wong Wai Tuck,et al.  An Exploration of Poisoning Attacks on Data-Based Decision Making , 2022, GameSec.

[101]  S. Sanner,et al.  An Exact Symbolic Reduction of Linear Smart Predict+Optimize to Mixed Integer Linear Programming , 2022, ICML.

[102]  A. Perrault,et al.  Decision-Focused Learning without Decision-Making: Learning Locally Optimized Decision Losses , 2022, NeurIPS.

[103]  A. Perrault,et al.  Learning MDPs from Features: Predict-Then-Optimize for Sequential Decision Making by Reinforcement Learning , 2021, NeurIPS.

[104]  Takuro Fukunaga,et al.  End-to-End Learning for Prediction and Optimization with Gradient Boosting , 2020, ECML/PKDD.

[105]  Dick den Hertog,et al.  Bridging the gap between predictive and prescriptive analytics-new optimization methodology needed , 2016 .

[106]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[107]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[108]  Deeparnab Chakrabarty,et al.  Knapsack Problems , 2008 .

[109]  Bart Selman,et al.  Satisfiability Solvers , 2008, Handbook of Knowledge Representation.

[110]  Yuval Rabani,et al.  Linear Programming , 2007, Handbook of Approximation Algorithms and Metaheuristics.

[111]  Toby Walsh,et al.  Handbook of Constraint Programming , 2006, Handbook of Constraint Programming.

[112]  A. Nemirovski Advances in convex optimization : conic programming , 2005 .

[113]  A. Ruszczynski Stochastic Programming Models , 2003 .

[114]  Hanif D. Sherali,et al.  Linear Programming and Network Flows , 1977 .