Model Order Reduction of Nonlinear Dynamical Systems

Author(s): Gu, Chenjie | Advisor(s): Roychowdhury, Jaijeet | Abstract: Higher-level representations (macromodels, reduced-order models) abstract away unnecessary implementation details and model only important system properties such as functionality. This methodology -- well-developed for linear systems and digital (Boolean) circuits -- is not mature for general nonlinear systems (such as analog/mixed-signal circuits). Questions arise regarding abstracting/macromodeling nonlinear dynamical systems: What are ``important'' system properties to preserve in the macromodel? What is the appropriate representation of the macromodel? What is the general algorithmic framework to develop a macromodel? How to automatically derive a macromodel from a white-box/black-box model? This dissertation presents techniques for solving the problem of macromodeling nonlinear dynamical systems by trying to answer these questions. We formulate the nonlinear model order reduction problem as an optimization problem and present a general nonlinear projection framework that encompasses previous linear projection-based techniques as well as the techniques developed in this dissertation. We illustrate that nonlinear projection is natural and appropriate for reducing nonlinear systems, and can achieve more compact and accurate reduced models than linear projection.The first method, ManiMOR, is a direct implementation of the nonlinear projection framework. It generates a nonlinear reduced model by projection on a general-purpose nonlinear manifold. The proposed manifold can be proven to capture important system dynamics such as DC and AC responses. We develop numerical methods that alleviates the computational cost of the reduced model which is otherwise too expensive to make the reduced order model of any value compared to the full model.The second method, QLMOR, transforms the full model to a canonical QLDAE representation and performs Volterra analysis to derive a reduced model. We develop an algorithm that can mechanically transform a set of nonlinear differential equations to another set of equivalent nonlinear differential equations that involve only quadratic terms of state variables, and therefore it avoids any problem brought by previous Taylor-expansion-based methods. With the QLDAE representation, we develop the corresponding model order reduction algorithm that extends and generalizes previously-developed Volterra-based technique.The third method, NTIM, derives a macromodel that specifically captures timing/phase responses of a nonlinear system. We rigorously define the phase response for a non-autonomous system,and derive the dynamics of the phase response. The macromodel emerges as a scalar, nonlinear time-varying differential equation that can be computed by performing Floquet analysis of the full model. With the theory developed, we also present efficient numerical methods to compute the macromodel.The fourth method, DAE2FSM, considers a slightly different problem -- finite state machine abstraction of continuous dynamical systems. We present an algorithm that learns a Mealy machine from a set of differential equations from its input-output trajectories. The algorithm explores the state space in a smart way so that it can identify the underlying finite state machine using very few information about input-output trajectories.

[1]  D Garfinkel,et al.  Computer applications to biochemical kinetics. , 1970, Annual review of biochemistry.

[2]  Jaijeet S. Roychowdhury Reduced-order modelling of time-varying systems , 1999, Proceedings of the ASP-DAC '99 Asia and South Pacific Design Automation Conference 1999 (Cat. No.99EX198).

[3]  Daniel Boley Krylov space methods on state-space control models , 1994 .

[4]  D. Hansel,et al.  Phase Dynamics for Weakly Coupled Hodgkin-Huxley Neurons , 1993 .

[5]  George J. Pappas,et al.  Hybrid systems: computation and abstraction , 1998 .

[6]  Lawrence T. Pileggi,et al.  Parameterized interconnect order reduction with explicit-and-implicit multi-parameter moment matching for inter/intra-die variations , 2005, ICCAD-2005. IEEE/ACM International Conference on Computer-Aided Design, 2005..

[7]  Eugene M. Izhikevich,et al.  Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting , 2006 .

[8]  Jaijeet Roychowdhury,et al.  Reduced-order modeling of time-varying systems , 1999 .

[9]  Jan G. Korvink,et al.  Model Order Reduction for Large Scale Engineering Models Developed in ANSYS , 2004, PARA.

[10]  Stefano Stramigioli,et al.  Energy-based model-reduction of nonholonomic mechanical systems , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[11]  John M. Lee Introduction to Smooth Manifolds , 2002 .

[12]  George Em Karniadakis,et al.  A low-dimensional model for simulating three-dimensional cylinder flow , 2002, Journal of Fluid Mechanics.

[13]  Eric James Grimme,et al.  Krylov Projection Methods for Model Reduction , 1997 .

[14]  B. Krogh,et al.  Towards formal verification of analog designs , 2004, ICCAD 2004.

[15]  Joel R. Phillips,et al.  First steps towards SAT-based formal analog verification , 2009, 2009 IEEE/ACM International Conference on Computer-Aided Design - Digest of Technical Papers.

[16]  Jian-Ming Jin,et al.  Model-order reduction of nonlinear models of electromagnetic phased-array hyperthermia , 2003, IEEE Transactions on Biomedical Engineering.

[17]  Jan G. Korvink,et al.  Efficient optimization of transient dynamic problems in MEMS devices using model order reduction , 2005 .

[18]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[19]  Ronald A. Rohrer,et al.  Electronic Circuit and System Simulation Methods , 1994 .

[20]  Jan G. Korvink,et al.  Review: Automatic Model Reduction for Transient Simulation of MEMS‐based Devices , 2002 .

[21]  D Lebiedz,et al.  Computing minimal entropy production trajectories: an approach to model reduction in chemical kinetics. , 2004, The Journal of chemical physics.

[22]  Rob A. Rutenbar,et al.  OASYS: a framework for analog circuit synthesis , 1989, Proceedings., Second Annual IEEE ASIC Seminar and Exhibit,.

[23]  J. Peraire,et al.  Balanced Model Reduction via the Proper Orthogonal Decomposition , 2002 .

[24]  Eric L Haseltine,et al.  Two classes of quasi-steady-state model reductions for stochastic kinetics. , 2007, The Journal of chemical physics.

[25]  Peng Li,et al.  On behavioral model equivalence checking for large analog/mixed signal systems , 2010, 2010 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[26]  Jörg Raisch,et al.  Discrete approximation and supervisory control of continuous systems , 1998, IEEE Trans. Autom. Control..

[27]  Shinji Doi,et al.  Reduction of a model for an Onchidium pacemaker neuron , 1998, Biological Cybernetics.

[28]  Kenji Doya,et al.  Dimension Reduction of Biological Neuron Models by Artificial Neural Networks , 1994, Neural Computation.

[29]  Drew Endy,et al.  Rapid Characterization of Cellular Pathways Using Time-Varying Signals , 2005 .

[30]  Mark de Berg,et al.  Computational geometry: algorithms and applications , 1997 .

[31]  Rolf Schuhmann,et al.  Two-step Lanczos algorithm for model order reduction , 2002 .

[32]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[33]  K R Schneider,et al.  Model reduction by extended quasi-steady-state approximation , 2000, Journal of mathematical biology.

[34]  Alberto L. Sangiovanni-Vincentelli,et al.  Steady-state methods for simulating analog and microwave circuits , 1990, The Kluwer international series in engineering and computer science.

[35]  S. Wong,et al.  Near speed-of-light signaling over on-chip electrical interconnects , 2003 .

[36]  Chris J. Myers,et al.  Verification of Analog/Mixed-Signal Circuits Using Symbolic Methods , 2008, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[37]  W. Rugh Nonlinear System Theory: The Volterra / Wiener Approach , 1981 .

[38]  Takayasu Sakurai,et al.  Power distribution analysis of VLSI interconnects using model orderreduction , 2002, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[39]  T.J. Brazil,et al.  Behavioral modeling of RF power amplifiers based on pruned volterra series , 2004, IEEE Microwave and Wireless Components Letters.

[40]  Alper Demir,et al.  A reliable and efficient procedure for oscillator PPV computation, with phase noise macromodeling applications , 2003, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[41]  Brian Munsky,et al.  Reduction and solution of the chemical master equation using time scale separation and finite state projection. , 2006, The Journal of chemical physics.

[42]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[43]  A. R. Newton,et al.  Alpha-power law MOSFET model and its applications to CMOS inverter delay and other formulas , 1990 .

[44]  Ulrich Maas,et al.  Simplifying chemical kinetics: Intrinsic low-dimensional manifolds in composition space , 1992 .

[45]  Peng Li,et al.  NORM: compact model order reduction of weakly nonlinear systems , 2003, Proceedings 2003. Design Automation Conference (IEEE Cat. No.03CH37451).

[46]  Ning Dong,et al.  Piecewise polynomial nonlinear model reduction , 2003, Proceedings 2003. Design Automation Conference (IEEE Cat. No.03CH37451).

[47]  Scott A. Smolka,et al.  Efficient Modeling of Excitable Cells Using Hybrid Automata , 2005 .

[48]  H. Shichman,et al.  Modeling and simulation of insulated-gate field-effect transistor switching circuits , 1968 .

[49]  Jacob K. White,et al.  A TBR-based trajectory piecewise-linear algorithm for generating accurate low-order models for nonlinear analog circuits and MEMS , 2003, Proceedings 2003. Design Automation Conference (IEEE Cat. No.03CH37451).

[50]  I K Fodor,et al.  A Survey of Dimension Reduction Techniques , 2002 .

[51]  A. Cangellaris,et al.  Model-order reduction of finite-element approximations of passive electromagnetic devices including lumped electrical-circuit models , 2004, IEEE Transactions on Microwave Theory and Techniques.

[52]  Chenjie Gu,et al.  QLMOR: A new projection-based approach for nonlinear model order reduction , 2009, 2009 IEEE/ACM International Conference on Computer-Aided Design - Digest of Technical Papers.

[53]  Rolf Isermann,et al.  Identification of Dynamic Systems: An Introduction with Applications , 2010 .

[54]  Bo Cartling,et al.  Response Characteristics of a Low-Dimensional Model Neuron , 1996, Neural Computation.

[55]  M. Mavrovouniotis,et al.  Simplification of Mathematical Models of Chemical Reaction Systems. , 1998, Chemical reviews.

[56]  K. Lu,et al.  Guaranteed Stable Projection-Based Model Reduction for Indefinite and Unstable Linear Systems * ) , 2009 .

[57]  J. Marsden,et al.  A subspace approach to balanced truncation for model reduction of nonlinear control systems , 2002 .

[58]  Joel R. Phillips,et al.  Projection-based approaches for model reduction of weakly nonlinear, time-varying systems , 2003, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[59]  Yang Xu,et al.  Analog and RF circuit macromodels for system-level analysis , 2003, Proceedings 2003. Design Automation Conference (IEEE Cat. No.03CH37451).

[60]  L. Petzold,et al.  Model reduction for chemical kinetics: an optimization approach , 1999 .

[61]  George J. Pappas,et al.  Discrete abstractions of hybrid systems , 2000, Proceedings of the IEEE.

[62]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[63]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[64]  Peter Benner,et al.  Dimension Reduction of Large-Scale Systems , 2005 .

[65]  Jacob K. White,et al.  A multiparameter moment-matching model-reduction approach for generating geometrically parameterized interconnect performance models , 2004, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[66]  Antoine Girard,et al.  Hybridization methods for the analysis of nonlinear systems , 2007, Acta Informatica.

[67]  J. Korvink,et al.  MST MEMS model order reduction: Requirements and benchmarks , 2006 .

[68]  Lawrence T. Pileggi,et al.  Compact reduced-order modeling of weakly nonlinear analog and RF circuits , 2005, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[69]  Jens Hoppe,et al.  The dynamics of relativistic membranes. Reduction to 2-dimensional fluid dynamics , 1993 .

[70]  Hiroyuki Kitajima,et al.  Bifurcations in Morris-Lecar neuron model , 2006, Neurocomputing.

[71]  Jaijeet S. Roychowdhury,et al.  Macromodelling oscillators using Krylov-subspace methods , 2006, Asia and South Pacific Conference on Design Automation, 2006..

[72]  Siak Piang Lim,et al.  PROPER ORTHOGONAL DECOMPOSITION AND ITS APPLICATIONS – PART II: MODEL REDUCTION FOR MEMS DYNAMICAL ANALYSIS , 2002 .

[73]  Gilead Tadmor,et al.  System reduction strategy for Galerkin models of fluid flows , 2009 .

[74]  J. Marsden,et al.  Reduction and reconstruction for self-similar dynamical systems , 2002 .

[75]  Alper Demir Floquet theory and non‐linear perturbation analysis for oscillators with differential‐algebraic equations , 2000, Int. J. Circuit Theory Appl..

[76]  Lawrence T. Pileggi,et al.  Asymptotic waveform evaluation for timing analysis , 1990, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[77]  Roland W. Freund,et al.  Efficient linear circuit analysis by Pade´ approximation via the Lanczos process , 1994, EURO-DAC '94.

[78]  Robert K. Brayton,et al.  The Sparse Tableau Approach to Network Analysis and Design , 1971 .

[79]  G. A. Baker,et al.  Padé Approximants Second Edition: Extensions of Padé approximants , 1996 .

[80]  Ying Liu,et al.  Model order-reduction of RC(L) interconnect including variational analysis , 1999, DAC '99.

[81]  P. Ioannou,et al.  Accurate Low-Dimensional Approximation of the Linear Dynamics of Fluid Flow , 2001 .

[82]  Edda Klipp,et al.  Modelling the dynamics of the yeast pheromone pathway , 2004, Yeast.

[83]  Jaijeet S. Roychowdhury,et al.  Numerical Simulation and Modelling of Electronic and Biochemical Systems , 2009, Found. Trends Electron. Des. Autom..

[84]  J. Phillips,et al.  Model reduction of time-varying linear systems using approximate multipoint Krylov-subspace projectors , 1998, 1998 IEEE/ACM International Conference on Computer-Aided Design. Digest of Technical Papers (IEEE Cat. No.98CB36287).

[85]  J. Roychowdhury,et al.  Implementing nonlinear oscillator macromodels using Verilog-AMS for accurate prediction of injection locking behaviors of oscillators , 2005, BMAS 2005. Proceedings of the 2005 IEEE International Behavioral Modeling and Simulation Workshop, 2005..

[86]  Jaijeet S. Roychowdhury,et al.  Generalized nonlinear timing/phase macromodeling: Theory, numerical methods and applications , 2010, 2010 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[87]  Gene H. Golub,et al.  Matrix computations , 1983 .

[88]  Jacob K. White,et al.  A trajectory piecewise-linear approach to model order reduction and fast simulation of nonlinear circuits and micromachined devices , 2001, IEEE/ACM International Conference on Computer Aided Design. ICCAD 2001. IEEE/ACM Digest of Technical Papers (Cat. No.01CH37281).

[89]  Petar V. Kokotovic,et al.  Singular perturbations and order reduction in control theory - An overview , 1975, Autom..

[90]  Thomas Kailath,et al.  Linear Systems , 1980 .

[91]  Keqin Li,et al.  Integration Testing of Components Guided by Incremental State Machine Learning , 2006, Testing: Academic & Industrial Conference - Practice And Research Techniques (TAIC PART'06).

[92]  Zhuyin Ren,et al.  The invariant constrained equilibrium edge preimage curve method for the dimension reduction of chemical kinetics. , 2006, The Journal of chemical physics.

[93]  I. Elfadel,et al.  A block rational Arnoldi algorithm for multipoint passive model-order reduction of multiport RLC networks , 1997, ICCAD 1997.

[94]  G. Kerschen,et al.  The Method of Proper Orthogonal Decomposition for Dynamical Characterization and Order Reduction of Mechanical Systems: An Overview , 2005 .

[95]  Jin-Fa Lee,et al.  Multipoint Galerkin asymptotic waveform evaluation for model order reduction of frequency domain FEM electromagnetic radiation problems , 2001 .

[96]  D.E. Root,et al.  A behavioral modeling approach to nonlinear model-order reduction for RF/microwave ICs and systems , 2004, IEEE Transactions on Microwave Theory and Techniques.

[97]  Janet Roveda,et al.  Extended Krylov subspace method for reduced order analysis of linear circuits with multiple sources , 2000, Proceedings 37th Design Automation Conference.

[98]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[99]  Lawrence T. Pileggi,et al.  PRIMA: passive reduced-order interconnect macromodeling algorithm , 1998, 1997 Proceedings of IEEE International Conference on Computer Aided Design (ICCAD).

[100]  Sani R. Nassif,et al.  Modeling interconnect variability using efficient parametric model order reduction , 2005, Design, Automation and Test in Europe.

[101]  Wulfram Gerstner,et al.  Reduction of the Hodgkin-Huxley Equations to a Single-Variable Threshold Model , 1997, Neural Computation.

[102]  R. Murray,et al.  Model reduction for compressible flows using POD and Galerkin projection , 2004 .

[103]  John Guckenheimer,et al.  Reduction of a channel-based model for a stomatogastric ganglion LP neuron , 2004, Biological Cybernetics.

[104]  Jaijeet S. Roychowdhury,et al.  FSM model abstraction for analog/mixed-signal circuits by learning from I/O trajectories , 2011, 16th Asia and South Pacific Design Automation Conference (ASP-DAC 2011).

[105]  Karen Willcox,et al.  Model Reduction for Large-Scale Systems with High-Dimensional Parametric Input Space , 2008, SIAM J. Sci. Comput..

[106]  B. Razavi,et al.  A 10-Gb/s CMOS clock and data recovery circuit with a half-rate linear phase detector , 2001, IEEE J. Solid State Circuits.

[107]  Luca Daniel,et al.  A Piecewise-Linear Moment-Matching Approach to Parameterized Model-Order Reduction for Highly Nonlinear Systems , 2007, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[108]  M. Spivak A comprehensive introduction to differential geometry , 1979 .

[109]  Jack Poulson,et al.  Scientific computing , 2013, XRDS.

[110]  J. Marsden,et al.  Structure-preserving Model Reduction of Mechanical Systems , 2000 .

[111]  Jaijeet S. Roychowdhury,et al.  TP-PPV: piecewise nonlinear, time-shifted oscillator macromodel extraction for fast, accurate PLL simulation , 2006, ICCAD.

[112]  B. Moore Principal component analysis in linear systems: Controllability, observability, and model reduction , 1981 .

[113]  Henrik G. Dohlman,et al.  Pheromone Signaling Mechanisms in Yeast: A Prototypical Sex Machine , 2004, Science.

[114]  Robert E. Tarjan,et al.  A Linear-Time Algorithm for Testing the Truth of Certain Quantified Boolean Formulas , 1979, Inf. Process. Lett..

[115]  Ning Dong,et al.  Automated nonlinear macromodelling of output buffers for high-speed digital applications , 2005, Proceedings. 42nd Design Automation Conference, 2005..

[116]  E. Coddington,et al.  Theory of Ordinary Differential Equations , 1955 .

[117]  Kurt Keutzer,et al.  Logic Synthesis , 1994 .

[118]  Li Zhao,et al.  Electromagnetic model order reduction for system-level modeling , 1999 .

[119]  S. Sastry Nonlinear Systems: Analysis, Stability, and Control , 1999 .

[120]  Alexandre Megretski,et al.  A Quasi-Convex Optimization Approach to Parameterized Model Order Reduction , 2005, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[121]  Yiyu Shi,et al.  Fast analysis of structured power grid by triangularization based structure preserving model order reduction , 2006, 2006 43rd ACM/IEEE Design Automation Conference.

[122]  Jaijeet Roychowdhury,et al.  Model reduction via projection onto nonlinear manifolds, with applications to analog circuits and biochemical systems , 2008, ICCAD 2008.

[123]  Sani R. Nassif,et al.  Power grid reduction based on algebraic multigrid principles , 2003, DAC '03.

[124]  Eve Marder,et al.  Reduction of conductance-based neuron models , 1992, Biological Cybernetics.

[125]  A. Demir,et al.  Phase noise in oscillators: a unifying theory and numerical methods for characterization , 2000 .

[126]  Sung-Mo Kang,et al.  Model-order reduction of nonlinear MEMS devices through arclength-based Karhunen-Loeve decomposition , 2001, ISCAS 2001. The 2001 IEEE International Symposium on Circuits and Systems (Cat. No.01CH37196).

[127]  Gunnar Cedersund,et al.  Reduction of a biochemical model with preservation of its basic dynamic properties , 2006, The FEBS journal.

[128]  Ernest S. Kuh,et al.  On projection-based algorithms for model-order reduction of interconnects , 2002 .

[129]  Michael Beitelschmidt,et al.  Comparison of model reduction techniques for large mechanical systems , 2008 .

[130]  Bradley N. Bond,et al.  Stability-preserving model reduction for linear and nonlinear systems arising in analog circuit applications , 2010 .

[131]  Scott Little,et al.  Efficient Modeling and Verification of Analog/Mixed Signal Circuits Using Labeled Hybrid Petri nets , 2008 .