On Joint Learning for Solving Placement and Routing in Chip Design

For its advantage in GPU acceleration and less dependency on human experts, machine learning has been an emerging tool for solving the placement and routing problems, as two critical steps in modern chip design flow. Being still in its early stage, there are fundamental issues: scalability, reward design, and end-toend learning paradigm etc. To achieve end-to-end placement learning, we first propose a joint learning method termed by DeepPlace for the placement of macros and standard cells, by the integration of reinforcement learning with a gradient based optimization scheme. To further bridge the placement with the subsequent routing task, we also develop a joint learning approach via reinforcement learning to fulfill both macro placement and routing, which is called DeepPR. One key design in our (reinforcement) learning paradigm involves a multi-view embedding model to encode both global graph level and local node level information of the input macros. Moreover, the random network distillation is devised to encourage exploration. Experiments on public chip design benchmarks show that our method can effectively learn from experience and also provides intermediate placement for the post standard cell placement, within few hours for training.

[1]  Melvin A. Breuer,et al.  A class of min-cut placement algorithms , 1988, DAC '77.

[2]  Shinn-Ying Ho,et al.  An orthogonal simulated annealing algorithm for large floorplanning problems , 2004, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[3]  Hua Xiang,et al.  DRC Hotspot Prediction at Sub-10nm Process Nodes Using Customized Convolutional Network , 2020, ISPD.

[4]  Andrew B. Kahng,et al.  BEOL stack-aware routability prediction from placement using data mining techniques , 2016, 2016 IEEE 34th International Conference on Computer Design (ICCD).

[5]  Patrick H. Madden,et al.  Recursive bisection placement: feng shui 5.0 implementation details , 2005, ISPD '05.

[6]  Wei-Kai Cheng,et al.  Evaluation of routability-driven macro placement with machine-learning technique , 2018, 2018 7th International Symposium on Next Generation Electronics (ISNE).

[7]  Andrew B. Kahng,et al.  Implementation and extensibility of an analytic placer , 2004, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[8]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[9]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[10]  Minsik Cho,et al.  BoxRouter 2.0: architecture and implementation of a hybrid and robust global router , 2007, 2007 IEEE/ACM International Conference on Computer-Aided Design.

[11]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[12]  C. Christopher Columbus,et al.  A Novel B*tree Crossover-Based Simulated Annealing Algorithm for Combinatorial Optimization in VLSI Fixed-Outline Floorplans , 2020, Circuits Syst. Signal Process..

[13]  Gi-Joon Nam,et al.  The ISPD2005 placement contest and benchmark suite , 2005, ISPD '05.

[14]  R. M. Mattheyses,et al.  A Linear-Time Heuristic for Improving Network Partitions , 1982, 19th Design Automation Conference.

[15]  Robert K. Brayton,et al.  A force-directed maze router , 2001, IEEE/ACM International Conference on Computer Aided Design. ICCAD 2001. IEEE/ACM Digest of Technical Papers (Cat. No.01CH37281).

[16]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[17]  Levent Burak Kara,et al.  Placement in Integrated Circuits using Cyclic Reinforcement Learning and Simulated Annealing , 2020, ArXiv.

[18]  Peter Spindler,et al.  Fast and Accurate Routing Demand Estimation for Efficient Routability-driven Placement , 2007, 2007 Design, Automation & Test in Europe Conference & Exhibition.

[19]  Chak-Kuen Wong,et al.  New algorithms for the rectilinear Steiner tree problem , 1990, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[20]  Levent Burak Kara,et al.  A Deep Reinforcement Learning Approach for Global Routing , 2019, ArXiv.

[21]  Chung-Kuan Cheng,et al.  ePlace: Electrostatics-Based Placement Using Fast Fourier Transform and Nesterov's Method , 2015, TODE.

[22]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[23]  Yao-Wen Chang,et al.  NTUplace3: An Analytical Placer for Large-Scale Mixed-Size Designs With Preplaced Blocks and Density Constraints , 2008, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[24]  Ulf Schlichtmann,et al.  Kraftwerk2—A Fast Force-Directed Quadratic Placement Approach Using an Accurate Net Model , 2008, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[25]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[26]  Christoph Albrecht,et al.  Global routing by new approximation algorithms for multicommodityflow , 2001, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[27]  Sachin S. Sapatnekar,et al.  A survey on multi-net global routing for integrated circuits , 2001, Integr..

[28]  Andrew B. Kahng,et al.  RePlAce: Advancing Solution Quality and Routability Validation in Global Placement , 2019, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[29]  Jiaqi Gu,et al.  DREAMPlace: Deep Learning Toolkit-Enabled GPU Acceleration for Modern VLSI Placement , 2020, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[30]  Amos J. Storkey,et al.  Exploration by Random Network Distillation , 2018, ICLR.

[31]  Quoc V. Le,et al.  A graph placement methodology for fast chip design , 2021, Nature.

[32]  F. Marquardt,et al.  Quantum circuit optimization with deep reinforcement learning , 2021, 2103.07585.

[33]  Haoxing Ren,et al.  Standard Cell Routing with Reinforcement Learning and Genetic Algorithm in Advanced Technology Nodes , 2021, 2021 26th Asia and South Pacific Design Automation Conference (ASP-DAC).