Tensor Computation: A New Framework for High-Dimensional Problems in EDA

Many critical electronic design automation (EDA) problems suffer from the curse of dimensionality, i.e., the very fast-scaling computational burden produced by large number of parameters and/or unknown variables. This phenomenon may be caused by multiple spatial or temporal factors (e.g., 3-D field solvers discretizations and multirate circuit simulation), nonlinearity of devices and circuits, large number of design or optimization parameters (e.g., full-chip routing/placement and circuit sizing), or extensive process variations (e.g., variability /reliability analysis and design for manufacturability). The computational challenges generated by such high-dimensional problems are generally hard to handle efficiently with traditional EDA core algorithms that are based on matrix and vector computation. This paper presents “tensor computation” as an alternative general framework for the development of efficient EDA algorithms and tools. A tensor is a high-dimensional generalization of a matrix and a vector, and is a natural choice for both storing and solving efficiently high-dimensional EDA problems. This paper gives a basic tutorial on tensors, demonstrates some recent examples of EDA applications (e.g., nonlinear circuit modeling and high-dimensional uncertainty quantification), and suggests further open EDA problems where the use of tensor computation could be of advantage.

[1]  Bart Vandereycken,et al.  Low-rank tensor completion by Riemannian optimization , 2014 .

[2]  A. Kibangou,et al.  Nonlinear system modeling and identification using Volterra‐PARAFAC models , 2012 .

[3]  Yang Zhang,et al.  Compact model order reduction of weakly nonlinear systems by associated transform , 2016, Int. J. Circuit Theory Appl..

[4]  Ngai Wong,et al.  Weakly nonlinear circuit analysis based on fast multidimensional inverse Laplace transform , 2012, 17th Asia and South Pacific Design Automation Conference.

[5]  Lijun Jiang,et al.  STAVES: Speedy tensor-aided Volterra-based electronic simulator , 2015, 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[6]  Luís Miguel Silveira,et al.  Resampling Plans for Sample Point Selection in Multipoint Model-Order Reduction , 2006, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[7]  Sarma B. K. Vrudhula,et al.  Hermite Polynomial Based Interconnect Analysis in the Presence of Process Variations , 2006, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[8]  S. Weinzierl Introduction to Monte Carlo methods , 2000, hep-ph/0006269.

[9]  Giovanni Marucci,et al.  Stochastic testing simulator for integrated circuits and MEMS: Hierarchical and sparse techniques , 2014, Proceedings of the IEEE 2014 Custom Integrated Circuits Conference.

[10]  Luca Daniel,et al.  Variation-aware interconnect extraction using statistical moment preserving model order reduction , 2010, DATE 2010.

[11]  Xin Li,et al.  Finding Deterministic Solution From Underdetermined Equation: Large-Scale Performance Variability Modeling of Analog/RF Circuits , 2010, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[12]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Ivan V. Oseledets,et al.  Fast Multidimensional Convolution in Low-Rank Tensor Formats via Cross Approximation , 2015, SIAM J. Sci. Comput..

[14]  Stefano Grivet-Talocia,et al.  Passivity enforcement via perturbation of Hamiltonian matrices , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[15]  Ngai Wong,et al.  Model Reduction and Simulation of Nonlinear Circuits via Tensor Decomposition , 2015, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[16]  Zhan Su,et al.  Uncertainty quantification of silicon photonic devices with correlated and non-Gaussian random parameters. , 2015, Optics express.

[17]  Fabio Nobile,et al.  A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data , 2007, SIAM Rev..

[18]  Sani R. Nassif,et al.  Multigrid-like technique for power grid analysis , 2001, IEEE/ACM International Conference on Computer Aided Design. ICCAD 2001. IEEE/ACM Digest of Technical Papers (Cat. No.01CH37281).

[19]  Michel S. Nakhla,et al.  Fast Variability Analysis of General Nonlinear Circuits Using Decoupled Polynomial Chaos , 2015, IEEE Transactions on Components, Packaging and Manufacturing Technology.

[20]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[21]  Xiu Yang,et al.  Enabling High-Dimensional Hierarchical Uncertainty Quantification by ANOVA and Tensor-Train Decomposition , 2014, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[22]  Jaijeet Roychowdhury,et al.  Efficient multi-tone distortion analysis of analog integrated circuits , 1995, Proceedings of the IEEE 1995 Custom Integrated Circuits Conference.

[23]  Mattan Kamon,et al.  FASTHENRY: a multipole-accelerated 3-D inductance extraction program , 1994 .

[24]  SchneiderReinhold,et al.  The Alternating Linear Scheme for Tensor Optimization in the Tensor Train Format , 2012 .

[25]  Tao Zhang,et al.  Fast 3-D Thermal Simulation for Integrated Circuits With Domain Decomposition Method , 2013, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[26]  Stephen P. Boyd,et al.  Optimal design of a CMOS op-amp via geometric programming , 2001, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[27]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[28]  Alberto L. Sangiovanni-Vincentelli,et al.  Steady-state methods for simulating analog and microwave circuits , 1990, The Kluwer international series in engineering and computer science.

[29]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[30]  Lieven De Lathauwer,et al.  Optimization-Based Algorithms for Tensor Decompositions: Canonical Polyadic Decomposition, Decomposition in Rank-(Lr, Lr, 1) Terms, and a New Generalization , 2013, SIAM J. Optim..

[31]  Lawrence T. Pileggi,et al.  PRIMA: passive reduced-order interconnect macromodeling algorithm , 1997, ICCAD 1997.

[32]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[33]  Mattan Kamon,et al.  FastHenry: A Multipole-Accelerated 3-D Inductance Extraction Program , 1993, 30th ACM/IEEE Design Automation Conference.

[34]  Jacob K. White,et al.  Efficient Steady-State Analysis Based on Matrix-Free Krylov-Subspace Methods , 1995, 32nd Design Automation Conference.

[35]  Luca Daniel,et al.  Stochastic Testing Method for Transistor-Level Uncertainty Quantification Based on Generalized Polynomial Chaos , 2013, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[36]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[37]  Chenjie Gu,et al.  QLMOR: A Projection-Based Nonlinear Model Order Reduction Approach Using Quadratic-Linear Representation of Nonlinear Systems , 2011, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[38]  A. Demir,et al.  Phase noise in oscillators: a unifying theory and numerical methods for characterization , 2000 .

[39]  Luís Miguel Silveira,et al.  A convex programming approach for generating guaranteed passive approximations to tabulated frequency-data , 2004, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[40]  Ngai Wong,et al.  STORM: A nonlinear model order reduction method via symmetric tensor decomposition , 2016, 2016 21st Asia and South Pacific Design Automation Conference (ASP-DAC).

[41]  Pinaki Mazumder,et al.  VLSI cell placement techniques , 1991, CSUR.

[42]  Ali M. Niknejad,et al.  BSIM — Industry standard compact MOSFET models , 2012, 2012 Proceedings of the ESSCIRC (ESSCIRC).

[43]  H. Bungartz,et al.  Sparse grids , 2004, Acta Numerica.

[44]  Dongbin Xiu,et al.  High-Order Collocation Methods for Differential Equations with Random Inputs , 2005, SIAM J. Sci. Comput..

[45]  Gérard Favier,et al.  Identification of PARAFAC-Volterra cubic models using an Alternating Recursive Least Squares algorithm , 2004, 2004 12th European Signal Processing Conference.

[46]  Luís Miguel Silveira,et al.  Guaranteed passive balancing transformations for model order reduction , 2002, DAC '02.

[47]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[48]  Ivan V. Oseledets,et al.  Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition , 2014, ICLR.

[49]  Anton van den Hengel,et al.  Semidefinite Programming , 2014, Computer Vision, A Reference Guide.

[50]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[51]  Visa Koivunen,et al.  Sequential Unfolding SVD for Tensors With Applications in Array Signal Processing , 2009, IEEE Transactions on Signal Processing.

[52]  Johan Håstad,et al.  Tensor Rank is NP-Complete , 1989, ICALP.

[53]  Jacob K. White,et al.  A precorrected-FFT method for electrostatic analysis of complicated 3-D structures , 1997, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[54]  Joel R. Phillips,et al.  Projection-based approaches for model reduction of weakly nonlinear, time-varying systems , 2003, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[55]  Jaijeet Roychowdhury,et al.  Reduced-order modeling of time-varying systems , 1999 .

[56]  Li Yu,et al.  Remembrance of transistors past: Compact model parameter extraction using bayesian inference and incomplete new measurements , 2014, 2014 51st ACM/EDAC/IEEE Design Automation Conference (DAC).

[57]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Jacob K. White,et al.  Geometrically parameterized interconnect performance models for interconnect synthesis , 2002, ISPD '02.

[59]  Tsui-Wei Weng,et al.  A big-data approach to handle process variations: Uncertainty quantification by tensor recovery , 2016, 2016 IEEE 20th Workshop on Signal and Power Integrity (SPI).

[60]  T. Aprille,et al.  Steady-state analysis of nonlinear circuits with periodic inputs , 1972 .

[61]  Paolo Maffezzoni,et al.  Efficient Uncertainty Quantification for the Periodic Steady State of Forced and Autonomous Circuits , 2013, IEEE Transactions on Circuits and Systems II: Express Briefs.

[62]  Nico Vervliet,et al.  Breaking the Curse of Dimensionality Using Decompositions of Incomplete Tensors: Tensor-based scientific computing in big data analysis , 2014, IEEE Signal Processing Magazine.

[63]  Liqing Zhang,et al.  Bayesian Sparse Tucker Models for Dimension Reduction and Tensor Completion , 2015, ArXiv.

[64]  Tsui-Wei Weng,et al.  A Big-Data Approach to Handle Many Process Variations: Tensor Recovery and Applications , 2016, ArXiv.

[65]  Reinhold Schneider,et al.  The Alternating Linear Scheme for Tensor Optimization in the Tensor Train Format , 2012, SIAM J. Sci. Comput..

[66]  Wei Cai,et al.  Optimization of a GPU Implementation of Multi-Dimensional RF Pulse Design Algorithm , 2011, 2011 5th International Conference on Bioinformatics and Biomedical Engineering.

[67]  Roland Pulch Modelling and simulation of autonomous oscillators with random parameters , 2011, Math. Comput. Simul..

[68]  T. Aprille,et al.  A computer algorithm to determine the steady-state response of nonlinear oscillators , 1972 .

[69]  Ibrahim M. Elfadel,et al.  Uncertainty quantification for integrated circuits: Stochastic spectral methods , 2013, 2013 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[70]  Alberto L. Sangiovanni-Vincentelli,et al.  An envelope-following method for the efficient transient simulation of switching power and filter circuits , 1988, [1988] IEEE International Conference on Computer-Aided Design (ICCAD-89) Digest of Technical Papers.

[71]  Alexandre Megretski,et al.  A Quasi-Convex Optimization Approach to Parameterized Model Order Reduction , 2005, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[72]  G KoldaTamara,et al.  Tensor Decompositions and Applications , 2009 .

[73]  Luca Benini,et al.  Policy optimization for dynamic power management , 1998, Proceedings 1998 Design and Automation Conference. 35th DAC. (Cat. No.98CH36175).

[74]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[75]  Luca Daniel,et al.  Parameterized model order reduction of nonlinear dynamical systems , 2005, ICCAD-2005. IEEE/ACM International Conference on Computer-Aided Design, 2005..

[76]  Ramachandra Achar,et al.  Fast variability analysis of general nonlinear circuits using decoupled polynomial chaos , 2014, 2014 IEEE 18th Workshop on Signal and Power Integrity (SPI).

[77]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[78]  Sarma B. K. Vrudhula,et al.  Stochastic analysis of interconnect performance in the presence of process variations , 2004, IEEE/ACM International Conference on Computer Aided Design, 2004. ICCAD-2004..

[79]  E. Bedrosian,et al.  The output properties of Volterra systems (nonlinear systems with memory) driven by harmonic and Gaussian inputs , 1971 .

[80]  Lawrence T. Pileggi,et al.  PRIMA: passive reduced-order interconnect macromodeling algorithm , 1998, 1997 Proceedings of IEEE International Conference on Computer Aided Design (ICCAD).

[81]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[82]  Rob A. Rutenbar,et al.  Virtual Probe: A Statistical Framework for Low-Cost Silicon Characterization of Nanoscale Integrated Circuits , 2011, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[83]  Jacob K. White,et al.  FastCap: a multipole accelerated 3-D capacitance extraction program , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[84]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[85]  E. Tyrtyshnikov,et al.  TT-cross approximation for multidimensional arrays , 2010 .

[86]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[87]  Ngai Wong,et al.  Autonomous Volterra Algorithm for Steady-State Analysis of Nonlinear Circuits , 2013, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[88]  David B. Dunson,et al.  Scalable Bayesian Low-Rank Decomposition of Incomplete Multiway Tensors , 2014, ICML.

[89]  Fabio Nobile,et al.  A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data , 2010, SIAM Rev..

[90]  Yan Li,et al.  Compact Modeling of Nonlinear Analog Circuits Using System Identification via Semidefinite Programming and Incremental Stability Certification , 2009, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[91]  J. Kruskal Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[92]  T. Dhaene,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON ADVANCED PACKAGING 1 Guaranteed Passive Parameterized , 2022 .

[93]  Jose C. Pedro,et al.  Multitone frequency-domain simulation of nonlinear circuits in large- and small-signal regimes , 1998 .

[94]  Xin Li,et al.  Robust analog/RF circuit design with projection-based posynomial modeling , 2004, IEEE/ACM International Conference on Computer Aided Design, 2004. ICCAD-2004..

[95]  Ngai Wong,et al.  A tensor-based volterra series black-box nonlinear system identification and simulation framework , 2016, 2016 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[96]  C. Loan The ubiquitous Kronecker product , 2000 .

[97]  Luca Daniel,et al.  A Piecewise-Linear Moment-Matching Approach to Parameterized Model-Order Reduction for Highly Nonlinear Systems , 2007, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[98]  Dries Vande Ginste,et al.  Stochastic Modeling of Nonlinear Circuits via SPICE-Compatible Spectral Equivalents , 2014, IEEE Transactions on Circuits and Systems I: Regular Papers.

[99]  L. Petzold An Efficient Numerical Method for Highly Oscillatory Ordinary Differential Equations , 1978 .

[100]  Hao Yu,et al.  A GPU-Accelerated Parallel Shooting Algorithm for Analysis of Radio Frequency and Microwave Integrated Circuits , 2015, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[101]  Jacob K. White,et al.  A multiparameter moment-matching model-reduction approach for generating geometrically parameterized interconnect performance models , 2004, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[102]  Lawrence T. Pileggi,et al.  Compact reduced-order modeling of weakly nonlinear analog and RF circuits , 2005, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[103]  Jacob K. White,et al.  Macromodel Generation for BioMEMS Components Using a Stabilized Balanced Truncation Plus Trajectory Piecewise-Linear Approach , 2006, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[104]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[105]  P. A. Cook Book Review: Nonlinear System Theory: The Volterra/Wiener Approach , 1983 .

[106]  Hans-Joachim Bungartz,et al.  Acta Numerica 2004: Sparse grids , 2004 .

[107]  Giovanni De Micheli,et al.  Synthesis and Optimization of Digital Circuits , 1994 .

[108]  Zhuo Feng,et al.  Multigrid on GPU: Tackling Power Grid Analysis on parallel SIMT platforms , 2008, 2008 IEEE/ACM International Conference on Computer-Aided Design.

[109]  Charlie Chung-Ping Chen,et al.  Efficient large-scale power grid analysis based on preconditioned Krylov-subspace iterative methods , 2001, Proceedings of the 38th Design Automation Conference (IEEE Cat. No.01CH37232).

[110]  Willy Sansen,et al.  Analog Circuit Design Optimization based on Symbolic Simulation and Simulated Annealing , 1989, ESSCIRC '89: Proceedings of the 15th European Solid-State Circuits Conference.

[111]  L. Nagel,et al.  SPICE (Simulation Program with Integrated Circuit Emphasis) , 1973 .

[112]  Fabio Nobile,et al.  A Sparse Grid Stochastic Collocation Method for Partial Differential Equations with Random Input Data , 2008, SIAM J. Numer. Anal..

[113]  J. Roychowdhury Analyzing circuits with widely separated time scales using numerical PDE methods , 2001 .

[114]  Luís Miguel Silveira,et al.  SPARE - a Scalable algorithm for passive, structure preserving, Parameter-Aware model order REduction , 2008, 2008 Design, Automation and Test in Europe.

[115]  W. Rugh Nonlinear System Theory: The Volterra / Wiener Approach , 1981 .

[116]  Fernando Corinto,et al.  Phase Noise and Noise Induced Frequency Shift in Stochastic Nonlinear Oscillators , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[117]  Ngai Wong,et al.  A Constructive Algorithm for Decomposing a Tensor into a Finite Sum of Orthonormal Rank-1 Terms , 2014, SIAM J. Matrix Anal. Appl..

[118]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[119]  Rob A. Rutenbar,et al.  Why Quasi-Monte Carlo is Better Than Monte Carlo or Latin Hypercube Sampling for Statistical Circuit Analysis , 2010, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[120]  BabuskaIvo,et al.  A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data , 2007 .

[121]  Gene H. Golub,et al.  Calculation of Gauss quadrature rules , 1967, Milestones in Matrix Computation.

[122]  Jacob K. White,et al.  A trajectory piecewise-linear approach to model order reduction and fast simulation of nonlinear circuits and micromachined devices , 2001, IEEE/ACM International Conference on Computer Aided Design. ICCAD 2001. IEEE/ACM Digest of Technical Papers (Cat. No.01CH37281).

[123]  Benjamin Barras,et al.  SPICE – Simulation Program with Integrated Circuit Emphasis , 2013 .

[124]  Qing Wang,et al.  Fast nonlinear model order reduction via associated transforms of high-order Volterra transfer functions , 2012, DAC Design Automation Conference 2012.

[125]  Albert E. Ruehli,et al.  The modified nodal approach to network analysis , 1975 .

[126]  Jason Cong,et al.  Performance optimization of VLSI interconnect layout , 1996, Integr..

[127]  Alexander Novikov,et al.  Tensorizing Neural Networks , 2015, NIPS.

[128]  Luca Daniel,et al.  Stable Reduced Models for Nonlinear Descriptor Systems Through Piecewise-Linear Approximation and Projection , 2009, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[129]  Ieee Circuits,et al.  IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems information for authors , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[130]  A. Semlyen,et al.  Rational approximation of frequency domain responses by vector fitting , 1999 .

[131]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[132]  Stephen P. Boyd,et al.  OPERA: optimization with ellipsoidal uncertainty for robust analog IC design , 2005, Proceedings. 42nd Design Automation Conference, 2005..