Accelerator-level parallelism

With the slowing of technology scaling, the only known way to further improve computer system performance under energy constraints is to employ hardware accelerators. Already today, many chips in mobile, edge and cloud computing concurrently employ multiple accelerators in what we call accelerator-level parallelism (ALP). For the needed benefits of ALP to spread to computer systems more broadly, we herein charge the community to develop better "best practices" for: targeting accelerators, managing accelerator concurrency, choreographing inter-accelerator communication, and productively programming accelerators.

[1]  Erik Lindholm,et al.  NVIDIA Tesla: A Unified Graphics and Computing Architecture , 2008, IEEE Micro.

[2]  Karthikeyan Sankaralingam,et al.  Heterogeneous Von Neumann/dataflow microprocessors , 2019, Commun. ACM.

[3]  David A. Patterson,et al.  A new golden age for computer architecture , 2019, Commun. ACM.

[4]  Rachata Ausavarungnirun,et al.  Google Workloads for Consumer Devices: Mitigating Data Movement Bottlenecks , 2018, ASPLOS.

[5]  Mark D. Hill,et al.  Gables: A Roofline Model for Mobile SoCs , 2019, 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[6]  William J. Dally,et al.  Domain-specific hardware accelerators , 2020, Commun. ACM.

[7]  David A. Patterson,et al.  A domain-specific architecture for deep neural networks , 2018, Commun. ACM.

[8]  Andrew A. Chien,et al.  The future of microprocessors , 2011, Commun. ACM.

[9]  Carole-Jean Wu,et al.  Machine Learning at Facebook: Understanding Inference at the Edge , 2019, 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[10]  Luca P. Carloni,et al.  Invited: The case for Embedded Scalable Platforms , 2016, 2016 53nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[11]  W. Daniel Hillis,et al.  Data parallel algorithms , 1986, CACM.

[12]  David A. Patterson,et al.  Computer Architecture: A Quantitative Approach , 1969 .

[13]  Vijay Janapa Reddi,et al.  Two Billion Devices and Counting , 2018, IEEE Micro.