Learning for Integer-Constrained Optimization through Neural Networks with Limited Training

In this paper, we investigate a neural network-based learning approach towards solving an integer-constrained programming problem using very limited training. To be specific, we introduce a symmetric and decomposed neural network structure, which is fully interpretable in terms of the functionality of its constituent components. By taking advantage of the underlying pattern of the integer constraint, as well as of the affine nature of the objective function, the introduced neural network offers superior generalization performance with limited training, as compared to other generic neural network structures that do not exploit the inherent structure of the integer constraint. In addition, we show that the introduced decomposed approach can be further extended to semi-decomposed frameworks. The introduced learning approach is evaluated via the classification/symbol detection task in the context of wireless communication systems where available training sets are usually limited. Evaluation results demonstrate that the introduced learning strategy is able to effectively perform the classification/symbol detection task in a wide variety of wireless channel environments specified by the 3GPP community.

[1]  Michael A. Saunders,et al.  Proximal Newton-type methods for convex optimization , 2012, NIPS.

[2]  Abdesselam Bouzerdoum,et al.  Neural network for quadratic optimization with bound constraints , 1993, IEEE Trans. Neural Networks.

[3]  D. Anderson,et al.  Algorithms for minimization without derivatives , 1974 .

[4]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[5]  Babak Hassibi,et al.  On the sphere-decoding algorithm I. Expected complexity , 2005, IEEE Transactions on Signal Processing.

[6]  Vishal Gupta,et al.  Data-driven robust optimization , 2013, Math. Program..

[7]  J. Zico Kolter,et al.  OptNet: Differentiable Optimization as a Layer in Neural Networks , 2017, ICML.

[8]  Ami Wiesel,et al.  Learning to Detect , 2018, IEEE Transactions on Signal Processing.

[9]  C. T. Kelley,et al.  A Locally-Biased form of the DIRECT Algorithm , 2001, J. Glob. Optim..

[10]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[11]  D. Bertsimas,et al.  Robust and Data-Driven Optimization: Modern Decision-Making Under Uncertainty , 2006 .

[12]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[13]  Raymond Hemmecke,et al.  Nonlinear Integer Programming , 2009, 50 Years of Integer Programming.

[14]  M. Powell A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation , 1994 .

[15]  Rajesh P. N. Rao,et al.  Bayesian brain : probabilistic approaches to neural coding , 2006 .

[16]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[17]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[18]  SapiroGuillermo,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2010 .

[19]  Gomes de Freitas,et al.  Bayesian methods for neural networks , 2000 .

[20]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[21]  Georges R. Harik,et al.  Foundations of Genetic Algorithms , 1997 .

[22]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[23]  Hugo Larochelle,et al.  RNADE: The real-valued neural autoregressive density-estimator , 2013, NIPS.

[24]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[25]  Rüdiger Schultz,et al.  Dual decomposition in stochastic integer programming , 1999, Oper. Res. Lett..

[26]  Jakob Hoydis,et al.  Adaptive Neural Signal Detection for Massive MIMO , 2019, IEEE Transactions on Wireless Communications.

[27]  T. Rowan Functional stability analysis of numerical algorithms , 1990 .

[28]  Babak Hassibi,et al.  On the sphere-decoding algorithm II. Generalizations, second-order statistics, and applications to communications , 2005, IEEE Transactions on Signal Processing.

[29]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[30]  Richard Lippmann,et al.  Neural Network Classifiers Estimate Bayesian a posteriori Probabilities , 1991, Neural Computation.

[31]  Jonathan Le Roux,et al.  Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures , 2014, ArXiv.

[32]  Gérard Dreyfus,et al.  Pairwise Neural Network Classifiers with Probabilistic Outputs , 1994, NIPS.

[33]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[34]  Sreeram Kannan,et al.  Communication Algorithms via Deep Learning , 2018, ICLR.