Generalized maximum entropy estimation

We consider the problem of estimating a probability distribution that maximizes the entropy while satisfying a finite number of moment constraints, possibly corrupted by noise. Based on duality of convex programming, we present a novel approximation scheme using a smoothed fast gradient method that is equipped with explicit bounds on the approximation error. We further demonstrate how the presented scheme can be used for approximating the chemical master equation through the zero-information moment closure method, and for an approximate dynamic programming approach in the context of constrained Markov decision processes with uncountable state and action spaces.

[1]  M. Sion.  On general minimax theorems , 1958 .

[2]  Onésimo Hernández-Lerma,et al.  Constrained Average Cost Markov Control Processes in Borel Spaces , 2003, SIAM J. Control. Optim..

[3]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[4]  E. Altman.  Constrained Markov Decision Processes , 1999 .

[5]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[6]  Renato Renner,et al.  Efficient Approximation of Quantum Channel Capacities , 2014, IEEE Transactions on Information Theory.

[7]  Benjamin Recht,et al.  A Tour of Reinforcement Learning: The View from Continuous Control , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[8]  H. Niederreiter,et al.  Quasi-Monte Carlo Methods , 2010 .

[9]  Daniel Kuhn,et al.  From Infinite to Finite Programs: Explicit Error Bounds with Applications to Approximate Dynamic Programming , 2018, SIAM J. Optim..

[10]  W. Ebeling.  Stochastic Processes in Physics and Chemistry , 1995 .

[11]  Yurii Nesterov,et al.  Double Smoothing Technique for Large-Scale Linearly Constrained Convex Optimization , 2012, SIAM J. Optim..

[12]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[13]  Frances Y. Kuo,et al.  High-dimensional integration: The quasi-Monte Carlo way*† , 2013, Acta Numerica.

[14]  Frances Y. Kuo,et al.  Lifting the Curse of Dimensionality , 2005 .

[15]  S. Pearson.  Moments , 2020, Narrative inquiry in bioethics.

[16]  Xuan Vinh Doan,et al.  Approximating integrals of multivariate exponentials: A moment approach , 2008, Oper. Res. Lett..

[17]  O. Hernández-Lerma,et al.  Further topics on discrete-time Markov control processes , 1999 .

[18]  Amos Golan.  Information and Entropy Econometrics - A Review and Synthesis , 2008 .

[19]  Kim C. Border,et al.  Infinite Dimensional Analysis: A Hitchhiker’s Guide , 1994 .

[20]  Rajmohan Rajaraman.  Randomized Rounding , 2016, Encyclopedia of Algorithms.

[21]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[22]  E. Anderson,et al.  Linear programming in infinite-dimensional spaces : theory and applications , 1987 .

[23]  Mohit Singh,et al.  Entropy, optimization and counting , 2013, STOC.

[24]  H. Umegaki.  CONDITIONAL EXPECTATION IN AN OPERATOR ALGEBRA, II , 1954 .

[25]  I. Csiszár.  $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[26]  João Pedro Hespanha,et al.  A Derivative Matching Approach to Moment Closure for the Stochastic Logistic Model , 2007, Bulletin of mathematical biology.

[27]  R. Baierlein.  Probability Theory: The Logic of Science , 2004 .

[28]  C. Gillespie.  Moment-closure approximations for mass-action models. , 2009, IET systems biology.

[29]  Andreas Hellander,et al.  Perspective: Stochastic algorithms for chemical kinetics. , 2013, The Journal of chemical physics.

[30]  Benjamin Van Roy,et al.  On Constraint Sampling in the Linear Programming Approach to Approximate Dynamic Programming , 2004, Math. Oper. Res..

[31]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[32]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[33]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[34]  Mohit Singh,et al.  A Randomized Rounding Approach to the Traveling Salesman Problem , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[35]  Arpit A. Almal,et al.  Lifting the Curse of Dimensionality , 2007 .

[36]  Daniel T Gillespie,et al.  Stochastic simulation of chemical kinetics. , 2007, Annual review of physical chemistry.

[37]  N. Akhiezer,et al.  The Classical Moment Problem and Some Related Questions in Analysis , 2020 .

[38]  Gautam Appa,et al.  Linear Programming in Infinite-Dimensional Spaces , 1989 .

[39]  Asuman E. Ozdaglar,et al.  Approximate Primal Solutions and Rate Analysis for Dual Subgradient Methods , 2008, SIAM J. Optim..

[40]  Yurii Nesterov,et al.  First-order methods of smooth convex optimization with inexact oracle , 2013, Mathematical Programming.

[41]  Peter Grünwald,et al.  Entropy concentration and the empirical coding game , 2008, ArXiv.

[42]  Miroslav Dudík,et al.  Maximum Entropy Density Estimation with Generalized Regularization and an Application to Species Distribution Modeling , 2007, J. Mach. Learn. Res..

[43]  L. Mead,et al.  Maximum entropy in the problem of moments , 1984 .

[44]  Nisheeth K. Vishnoi,et al.  Computing Maximum Entropy Distributions Everywhere , 2017, ArXiv.

[45]  A. Piunovskiy.  Optimal Control of Random Sequences in Problems with Constraints , 1997 .

[46]  Dimitri P. Bertsekas,et al.  Convex Optimization Theory , 2009 .

[47]  Darren J. Wilkinson.  Stochastic Modelling for Systems Biology , 2006 .

[48]  Henryk Wozniakowski,et al.  When Are Quasi-Monte Carlo Algorithms Efficient for High Dimensional Integrals? , 1998, J. Complex..

[49]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[50]  J. Lasserre.  Moments, Positive Polynomials And Their Applications , 2009 .

[51]  Stefan Richter,et al.  Computational complexity certification of gradient methods for real-time model predictive control , 2012 .

[52]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[53]  D. Sherrington.  Stochastic Processes in Physics and Chemistry , 1983 .

[54]  Amin Saberi,et al.  An O(log n/ log log n)-approximation algorithm for the asymmetric traveling salesman problem , 2010, SODA '10.

[55]  John Lygeros,et al.  Efficient Approximation of Channel Capacities , 2015, IEEE Transactions on Information Theory.

[56]  J.P. Hespanha,et al.  Lognormal Moment Closures for Biochemical Reactions , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[57]  Sheehan Olver,et al.  On the convergence rate of a modified Fourier series , 2009, Math. Comput..

[58]  Amin Saberi,et al.  An O(log n/ log log n)-approximation algorithm for the asymmetric traveling salesman problem , 2010, SODA '10.

[59]  Rida T. Farouki,et al.  The Bernstein polynomial basis: A centennial retrospective , 2012, Comput. Aided Geom. Des..

[60]  Yiannis N. Kaznessis,et al.  A closure scheme for chemical master equations , 2013, Proceedings of the National Academy of Sciences.

[61]  D. Gillespie.  A General Method for Numerically Simulating the Stochastic Time Evolution of Coupled Chemical Reactions , 1976 .