Rational approximation and its application to improving deep learning classifiers.

A rational approximation by a ratio of polynomial functions is a flexible alternative to polynomial approximation. In particular, rational functions exhibit accurate estimations to nonsmooth and non- Lipschitz functions, where polynomial approximations are not efficient. We prove that the optimisation problems appearing in the best uniform rational approximation are quasiconvex, and show how to use this fact for calculating the best approximation in a fast and efficient method. The paper presents a theoretical study of the arising optimisation problems and provides results of several numerical experiments. In all our computations, the algorithms terminated at optimal solutions. We apply our approximation as a preprocess step to deep learning classifiers and demonstrate that the classification accuracy is significantly improved compared to the classification of the raw signals.

[1]  Günther Nürnberger,et al.  Bivariate segment approximation and free knot splines: Research problems 96-4 , 1996 .

[2]  Yoel Shkolnisky,et al.  Matrix Chebyshev expansion and its application to eigenspaces recovery , 2015 .

[3]  Characterization theorem for best polynomial spline approximation with free knots , 2014 .

[4]  Nadezda Sukhorukova,et al.  Optimization-based features extraction for K-complex detection , 2014 .

[5]  Nadezda Sukhorukova,et al.  Linear least squares problems involving fixed knots polynomial splines and their singularity study , 2016, Appl. Math. Comput..

[6]  M. Sion.  On general minimax theorems , 1958 .

[7]  G. Nürnberger.  Approximation by Spline Functions , 1989 .

[8]  G. Meinardus.  Approximation of Functions: Theory and Numerical Methods , 1967 .

[9]  A. Rubinov.  Abstract Convexity and Global Optimization , 2000 .

[10]  Functions whose best rational Chebyshev approximations are polynomials , 1964 .

[11]  Günther Nürnberger,et al.  Algorithms for piecewise polynomials and splines with free knots , 1989 .

[12]  J. Crouzeix,et al.  Finite Alternation Theorems and a Constructive Approach to Piecewise Polynomial Approximation in Chebyshev Norm , 2020 .

[13]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[14]  Theodore J. Rivlin.  Polynomials of best uniform approximation to certain rational functions , 1962 .

[15]  R. Northrop.  Signals and Systems Analysis In Biomedical Engineering , 2003 .

[16]  Nicolas Hadjisavvas,et al.  An Appropriate Subdifferential for Quasiconvex Functions , 2002, SIAM J. Optim..

[17]  Siegfried Schaible,et al.  Handbook of Generalized Convexity and Generalized Monotonicity , 2005 .

[18]  Anthony Ralston,et al.  Rational chebyshev approximation by Remes' algorithms , 1965 .

[19]  Nadezda Sukhorukova,et al.  Convex optimisation-based methods for K-complex detection , 2015, Appl. Math. Comput..

[20]  D. Wulbert.  The Rational Approximation of Real Functions , 1978 .

[21]  Z. Roshan Zamir,et al.  Detection of epileptic seizure in EEG signals using linear least squares preprocessing , 2016, Comput. Methods Programs Biomed..

[22]  D. Braess.  Chebyshev approximation by spline functions with free knots , 1971 .

[23]  L. Trefethen,et al.  Barycentric-Remez algorithms for best polynomial approximation in the chebfun system , 2009 .

[24]  Lloyd N. Trefethen,et al.  Rational Minimax Approximation via Adaptive Barycentric Representations , 2017, SIAM J. Sci. Comput..

[25]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[26]  Radakovič.  The theory of approximation , 1932 .

[27]  K Lehnertz,et al.  Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: dependence on recording region and brain state. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[28]  L. Trefethen.  Approximation Theory and Approximation Practice (Other Titles in Applied Mathematics) , 2012 .

[29]  G. Lorentz,et al.  Constructive approximation : advanced problems , 1996 .

[30]  Roland W. Freund,et al.  Computing Fundamental Matrix Decompositions Accurately via the Matrix Sign Function in Two Iterations: The Power of Zolotarev's Functions , 2016, SIAM Rev..

[31]  Jean-Pierre Crouzeix.  Conditions for Convexity of Quasiconvex Functions , 1980, Math. Oper. Res..

[32]  E. Ya. Remez,et al.  General computational methods of Chebyshev approximation : the problems with linear real parameters , 1967 .

[33]  L. Schumaker.  Uniform Approximation by Chebyshev Spline Functions. II: Free Knots , 1968 .

[34]  Lloyd N. Trefethen,et al.  The AAA Algorithm for Rational Approximation , 2016, SIAM J. Sci. Comput..

[35]  Anthony Ralston,et al.  Mathematical Methods for Digital Computers , 1960 .

[36]  Nadezda Sukhorukova,et al.  Uniform Approximation by the Highest Defect Continuous Polynomial Splines: Necessary and Sufficient Optimality Conditions and Their Generalisations , 2010, J. Optim. Theory Appl..

[37]  Caleb C. Magruder,et al.  Least Squares Rational Approximation , 2018, 1811.12590.