The future of computing beyond Moore’s Law

Moore’s Law is a techno-economic model that has enabled the information technology industry to double the performance and functionality of digital electronics roughly every 2 years within a fixed cost, power and area. Advances in silicon lithography have enabled this exponential miniaturization of electronics, but, as transistors reach atomic scale and fabrication costs continue to rise, the classical technological driver that has underpinned Moore’s Law for 50 years is failing and is anticipated to flatten by 2025. This article provides an updated view of what a post-exascale system will look like and the challenges ahead, based on our most recent understanding of technology roadmaps. It also discusses the tapering of historical improvements, and how it affects options available to continue scaling of successors to the first exascale machine. Lastly, this article covers the many different opportunities and strategies available to continue computing performance improvements in the absence of historical technology drivers. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

[1]  Jeremy Hsu.  Nervana systems: Turning neural networks into a service [Resources_Startups] , 2016, IEEE Spectrum.

[2]  John Shalf,et al.  Computing beyond Moore's Law , 2015, Computer.

[3]  Neil Thompson,et al.  The Decline of Computers As a General Purpose Technology: Why Deep Learning and the End of Moore’s Law are Fragmenting Computing , 2018 .

[4]  J. P. Grossman,et al.  Anton 2: Raising the Bar for Performance and Programmability in a Special-Purpose Molecular Dynamics Supercomputer , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[5]  John Shalf,et al.  Trends in Data Locality Abstractions for HPC Systems , 2017, IEEE Transactions on Parallel and Distributed Systems.

[6]  D.A.B. Miller,et al.  Rationale and challenges for optical interconnects to electronic chips , 2000, Proceedings of the IEEE.

[7]  Gu-Yeon Wei,et al.  Co-designing accelerators and SoC interfaces using gem5-Aladdin , 2016, 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[8]  Dmitri E. Nikonov,et al.  Overview of Beyond-CMOS Devices and a Uniform Methodology for Their Benchmarking , 2013, Proceedings of the IEEE.

[9]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[10]  John Shalf,et al.  Challenges and opportunities in system-level evaluation of photonics , 2019, OPTO.

[11]  Jack Dongarra,et al.  Numerical algorithms for high-performance computational science , 2020, Philosophical Transactions of the Royal Society A.

[12]  Mark Horowitz,et al.  High-speed electrical signaling: overview and limitations , 1998, IEEE Micro.

[13]  Robert P. Colwell,et al.  The chip design game at the end of Moore's law , 2013, 2013 IEEE Hot Chips 25 Symposium (HCS).

[14]  Kunle Olukotun,et al.  Plasticine: A Reconfigurable Accelerator for Parallel Patterns , 2018, IEEE Micro.

[15]  Makoto Taiji,et al.  MDGRAPE-4: a special-purpose computer system for molecular dynamics simulations , 2014, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[16]  Hans Johansen,et al.  Software Productivity for Extreme-Scale Science , 2022 .

[17]  Samuel Williams,et al.  The Landscape of Parallel Computing Research: A View from Berkeley , 2006 .

[18]  Chris Mack,et al.  The Multiple Lives of Moore's Law , 2015, IEEE Spectrum.

[19]  Igor L. Markov,et al.  Limits on fundamental limits to computation , 2014, Nature.

[20]  John Shalf,et al.  Exascale Computing Trends: Adjusting to the "New Normal"' for Computer Architecture , 2013, Computing in Science & Engineering.

[21]  David A. B. Miller,et al.  Limit to the Bit-Rate Capacity of Electrical Interconnects from the Aspect Ratio of the System Architecture , 1997, J. Parallel Distributed Comput..

[22]  G.E. Moore,et al.  Cramming More Components Onto Integrated Circuits , 1998, Proceedings of the IEEE.

[23]  Keren Bergman.  Empowering Flexible and Scalable High Performance Architectures with Embedded Photonics , 2018, 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[24]  Mario Nemirovsky,et al.  Disaggregated Computing. An Evaluation of Current Trends for Datacentres , 2017, ICCS.

[25]  Hari Angepat,et al.  A cloud-scale acceleration architecture , 2016, 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[27]  Jason Taylor,et al.  Facebook's data center infrastructure: Open compute, disaggregated rack, and beyond , 2015, 2015 Optical Fiber Communications Conference and Exhibition (OFC).