High performance Monte Carlo simulation of ising model on TPU clusters

Large-scale deep learning benefits from an emerging class of AI accelerators. Some of these accelerators' designs are general enough for compute-intensive applications beyond AI and Cloud TPU is one such example. In this paper, we demonstrate a novel approach using TensorFlow on Cloud TPU to simulate the two-dimensional Ising Model. TensorFlow and Cloud TPU framework enable the simple and readable code to express the complicated distributed algorithm without compromising the performance. Our code implementation fits into a small Jupyter Notebook and fully utilizes Cloud TPU's efficient matrix operation and dedicated high speed inter-chip connection. The performance is highly competitive: it outperforms the best published benchmarks to our knowledge by 60% in single-core and 250% in multi-core with good linear scaling. When compared to Tesla V100 GPU, the single-core performance maintains a ~10% gain. We also demonstrate that using low precision arithmetic---bfloat16---does not compromise the correctness of the simulation results.

[1]  Sheldon X.-D. Tan,et al.  GPU Based Parallel Ising Computing for Combinatorial Optimization Problems in VLSI Physical Design. , 2018, 1807.10750.

[2]  Wolfgang Paul,et al.  GPU accelerated Monte Carlo simulation of the 2D and 3D Ising model , 2009, J. Comput. Phys..

[3]  K. Binder.  Finite size scaling analysis of ising model block distribution functions , 1981 .

[4]  Peter Virnau,et al.  Multi-GPU accelerated multi-spin Monte Carlo simulations of the 2D Ising model , 2010, Comput. Phys. Commun..

[5]  Marcelo A. Montemurro,et al.  FPGA Hardware Acceleration of Monte Carlo Simulations for the Ising Model , 2016, IEEE Transactions on Parallel and Distributed Systems.

[6]  L. Onsager.  Crystal statistics. I. A two-dimensional model with an order-disorder transition , 1944 .

[7]  Wolfgang J. Paul,et al.  System Architecture , 2016, Springer International Publishing.

[8]  Alan M. Ferrenberg,et al.  Pushing the limits of Monte Carlo simulations for the three-dimensional Ising model. , 2018, Physical review. E.

[9]  Massimo Bernaschi,et al.  A Performance Study of the 2D Ising Model on GPUs , 2019, Comput. Phys. Commun..

[10]  K. Binder,et al.  Monte Carlo Simulation in Statistical Physics , 1992, Graduate Texts in Physics.

[11]  E. Ising.  Beitrag zur Theorie des Ferromagnetismus , 1925 .

[12]  Tobias Preis,et al.  Fluctuation patterns in high-frequency financial asset returns , 2008 .

[13]  Alan M. Ferrenberg,et al.  Pushing the Limits of Monte Carlo Simulations for the 3d Ising Model , 2017 .

[14]  David Patterson,et al.  A domain-specific supercomputer for training deep neural networks , 2020, Commun. ACM.

[15]  Juan M. Corchado,et al.  An Ising Spin-Based Model to Explore Efficient Flexibility in Distributed Power Systems , 2018, Complex..

[16]  Wang,et al.  In-Datacenter Performance Analysis of a Tensor Processing UnitTM , .

[17]  Kurt Binder,et al.  Monte Carlo tests of renormalization-group predictions for critical phenomena in Ising models , 2001 .

[18]  Germinal Cocho,et al.  Cancer growth and metastasis as a metaphor of Go gaming: An Ising model approach , 2018, PloS one.

[19]  Kurt Binder,et al.  Finite size scaling analysis of ising model block distribution functions , 1981 .

[20]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.