A Revision of Relaxed Steepest Descent Method from the Dynamics on an Invariant Manifold

Based-on the ordinary differential equations defined on an invariant manifold, we propose a theoretical procedure to derive a Relaxed Steepest Descent Method (RSDM) for numerically solving an ill-posed system of linear equations when the data are polluted by random noise. The invariant manifold is defined in terms of a squared-residual-norm and a fictitious time-like variable, and in the final stage we can derive an iterative algorithm including a parameter, which is known as the relaxation parameter. Through a Hopf bifurcation, this parameter indeed plays a major role to switch the situation of slow convergence to a new situation with faster convergence. Several numerical examples, including the first-kind Fredholm integral equation and backward heat conduction problem, are examined and compared with exact solutions, revealing that the RSDM has superior computational efficiency and accuracy even for the highly ill-conditioned linear equations with a large noise imposed on the given data.

[1]  D. Luenberger.  The Gradient Projection Method Along Geodesics , 1972 .

[2]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[3]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[4]  M. Bartholomew-Biggs,et al.  Some effective methods for unconstrained optimization based on the solution of systems of ordinary differential equations , 1989 .

[5]  William H. Press,et al.  Numerical Recipes: FORTRAN , 1988 .

[6]  Alfred K. Louis,et al.  The instability of some gradient methods for ill-posed problems , 1990 .

[7]  W. Press,et al.  Numerical Recipes in Fortran: The Art of Scientific Computing.@@@Numerical Recipes in C: The Art of Scientific Computing. , 1994 .

[8]  A. Bloch.  Hamiltonian and Gradient Flows, Algorithms and Control , 1995 .

[9]  M. Hanke.  Conjugate gradient type methods for ill-posed problems , 1995 .

[10]  M. Golberg,et al.  Discrete projection methods for integral equations , 1996 .

[11]  U. Helmke,et al.  Optimization and Dynamical Systems , 1994, Proceedings of the IEEE.

[12]  K. Kunisch,et al.  Iterative choices of regularization parameters in linear inverse problems , 1998 .

[13]  Chein-Shan Liu,et al.  Intermittent transition to quasiperiodicity demonstrated via a circular differential equation , 2000 .

[14]  Yanfei Wang,et al.  Fast realization algorithms for determining regularization parameters in linear inverse problems , 2001 .

[15]  Chein-Shan Liu.  Cone of non-linear dynamical system and group preserving schemes , 2001 .

[16]  Jun Zou,et al.  An improved model function method for choosing regularization parameters in linear inverse problems , 2002 .

[17]  R. Adler,et al.  Newton's method on Riemannian manifolds and a geometric model for the human spine , 2002 .

[18]  Jacques Laminie,et al.  Differential equations and solution of linear systems , 2005, Numerical Algorithms.

[19]  E. Resmerita.  Regularization of ill-posed problems in Banach spaces: convergence rates , 2005 .

[20]  Chein-Shan Liu,et al.  Nonstandard Group-Preserving Schemes for Very Stiff Ordinary Differential Equations , 2005 .

[21]  A study of type I intermittency of a circular differential equation under a discontinuous right-hand side , 2007 .

[22]  A. G. Ramm,et al.  Solving ill-conditioned linear algebraic systems by the dynamical systems method , 2007, 0705.4074.

[23]  Yaguang Yang.  Globally Convergent Optimization Algorithms on Riemannian Manifolds: Uniform Framework for Unconstrained and Constrained Optimization , 2007 .

[24]  Pierre-Antoine Absil,et al.  Trust-Region Methods on Riemannian Manifolds , 2007, Found. Comput. Math..

[25]  Paul Tseng.  Control perspectives on numerical algorithms and matrix problems , 2008, Math. Comput..

[26]  Satya N. Atluri,et al.  A Novel Time Integration Method for Solving A Large System of Non-Linear Algebraic Equations , 2008 .

[27]  Chein-Shan Liu,et al.  A Time-Marching Algorithm for Solving Non-Linear Obstacle Problems with the Aid of an NCP-Function , 2008 .

[28]  A. G. Ramm,et al.  Dynamical Systems Gradient Method for Solving Ill-Conditioned Linear Algebraic Systems , 2008, 0802.3933.

[29]  P. Absil,et al.  An implicit trust-region method on Riemannian manifolds , 2008 .

[30]  Chein-Shan Liu,et al.  A Fictitious Time Integration Method for a Quasilinear Elliptic Boundary Value Problem, Defined in an Arbitrary Plane Domain , 2009 .

[31]  Satya N. Atluri,et al.  A Highly Accurate Technique for Interpolations Using Very High-Order Polynomials, and Its Applications to Some Ill-Posed Linear Problems , 2009 .

[32]  Chein-Shan Liu,et al.  A Novel Method for Solving the Cauchy Problem of Laplace Equation Using the Fictitious Time Integration Method , 2009 .

[33]  Chih-Wen Chang,et al.  A Fictitious Time Integration Method for Backward Advection-Dispersion Equation , 2009 .

[34]  Chein-Shan Liu.  A Fictitious Time Integration Method for the Burgers Equation , 2009 .

[35]  M. Li,et al.  A DISCREPANCY PRINCIPLE FOR THE SOURCE POINTS LOCATION IN USING THE MFS FOR SOLVING THE BHCP , 2009 .

[36]  Chein-Shan Liu,et al.  Applications of the Fictitious Time Integration Method Using a New Time-Like Function , 2009 .

[37]  Chein-Shan Liu,et al.  NOVEL METHODS FOR SOLVING SEVERELY ILL-POSED LINEAR EQUATIONS SYSTEM , 2009 .

[38]  Chein-Shan Liu,et al.  A Fictitious Time Integration Method for Solving Delay Ordinary Differential Equations , 2009 .

[39]  S. Atluri,et al.  On Solving the Ill-Conditioned System Ax=b: General-Purpose Conditioners Obtained From the Boundary-Collocation Solution of the Laplace Equation, Using Trefftz Expansions With Multiple Length Scales , 2009 .

[40]  Satya N. Atluri,et al.  A Fictitious Time Integration Method for the Numerical Solution of the Fredholm Integral Equation and for Numerical Differentiation of Noisy Data, and Its Relation to the Filter Theory , 2009 .

[41]  S. Atluri,et al.  Novel Algorithms Based on the Conjugate Gradient Method for Inverting Ill-Conditioned Matrices, and a New Regularization Method to Solve Ill-Posed Linear Systems , 2010 .

[42]  Chein-Shan Liu.  The Fictitious Time Integration Method to Solve the Space- and Time-Fractional Burgers Equations , 2010 .

[43]  Satya N. Atluri,et al.  Simple "Residual-Norm" Based Algorithms, for the Solution of a Large System of Non-Linear Algebraic Equations, which Converge Faster than the Newton's Method , 2011 .

[44]  Chein-Shan Liu.  The Method of Fundamental Solutions for Solving the Backward Heat Conduction Problem with Conditioning by a New Post-Conditioner , 2011 .