Network-on-Chip Microarchitecture-based Covert Channel in GPUs

As GPUs are becoming widely deployed in the cloud infrastructure to support different application domains, the security concerns of GPUs are becoming increasingly important. In particular, the support for multiprogramming in modern GPUs has led to new vulnerabilities since multiple kernels in a GPU can be executed at the same time. In this work, we propose a new microarchitectural timing covert channel for GPUs that can be established based on the shared, on-chip interconnect channels. We first reverse-engineer the organization of the on-chip networks in modern GPUs to understand the core placements throughout the GPU. The hierarchical organization of the GPU results in the sharing of interconnect bandwidth between neighboring cores. Based on this understanding, we identify how contention for the interconnect bandwidth can be exploited for a novel covert channel attack. We propose two types of interconnect-based covert channels that exploit the on-chip network hierarchy. Unlike cache-based covert channels, no states of the on-chip network need to be modified for communication in our interconnect-based covert channel and the impact of contention is very predictable. By exploiting the parallelism of GPUs, our proposed covert channel results in very high bandwidth – achieving approximately 24 Mbps of bandwidth on NVIDIA Volta GPUs and results in one of the highest known microarchitectural covert channel bandwidth.

[1]  Ying Gao,et al.  SurfNoC: a low latency and provably non-interfering approach to secure networks-on-chip , 2013, ISCA.

[2]  Joseph Zambreno,et al.  Increasing GPU throughput using kernel interleaved thread block scheduling , 2013, 2013 IEEE 31st International Conference on Computer Design (ICCD).

[3]  Mike O'Connor,et al.  Cache-Conscious Wavefront Scheduling , 2012, 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture.

[4]  Butler W. Lampson,et al.  A note on the confinement problem , 1973, CACM.

[5]  Craig A. Shue,et al.  Reporting Insider Threats via Covert Channels , 2013, 2013 IEEE Security and Privacy Workshops.

[6]  Nael B. Abu-Ghazaleh,et al.  Covert channels through branch predictors: a feasibility study , 2015, HASP@ISCA.

[7]  William J. Dally,et al.  Principles and Practices of Interconnection Networks , 2004 .

[8]  Yier Jin,et al.  Microarchitectural Minefields: 4K-Aliasing Covert Channel and Multi-Tenant Detection in Iaas Clouds , 2018, NDSS.

[9]  John Kim,et al.  iPAWS: Instruction-issue pattern-based adaptive warp scheduling for GPGPUs , 2016, 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[10]  Hoda Naghibijouybari,et al.  Leaky Buddies: Cross-Component Covert Channels on Integrated CPU-GPU Systems , 2020, 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA).

[11]  Wojciech Mazurczyk,et al.  Improving Hard Disk Contention-Based Covert Channel in Cloud Computing , 2014, 2014 IEEE Security and Privacy Workshops.

[12]  Mike O'Connor,et al.  Divergence-Aware Warp Scheduling , 2013, 2013 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[13]  Erik Lindholm,et al.  NVIDIA Tesla: A Unified Graphics and Computing Architecture , 2008, IEEE Micro.

[14]  Colin Percival CACHE MISSING FOR FUN AND PROFIT , 2005 .

[15]  Scott A. Mahlke,et al.  Mascar: Speeding up GPU warps by reducing memory pitstops , 2015, 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA).

[16]  Chia-Lin Yang,et al.  Improving GPGPU Performance via Cache Locality Aware Thread Block Scheduling , 2017, IEEE Computer Architecture Letters.

[17]  Konstantinos Markantonakis,et al.  Leaky Controller: Cross-VM Memory Controller Covert Channel on Multi-core Systems , 2020, SEC.

[18]  William J. Dally,et al.  Route packets, not wires: on-chip inteconnection networks , 2001, DAC '01.

[19]  David R. Kaeli,et al.  A complete key recovery timing attack on a GPU , 2016, 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[20]  Deborah K. Weisser,et al.  Age-based packet arbitration in large-radix k-ary n-cubes , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[21]  Simha Sethumadhavan,et al.  TimeWarp: Rethinking timekeeping and performance monitoring mechanisms to mitigate side-channel attacks , 2012, 2012 39th Annual International Symposium on Computer Architecture (ISCA).

[22]  Christopher W. Fletcher,et al.  Lord of the Ring(s): Side Channel Attacks on the CPU On-Chip Ring Interconnect Are Practical , 2021, USENIX Security Symposium.

[23]  John Kim,et al.  Throughput-Effective On-Chip Networks for Manycore Accelerators , 2010, 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture.

[24]  Matti A. Hiltunen,et al.  An exploration of L2 cache covert channels in virtualized environments , 2011, CCSW '11.

[25]  Ruby B. Lee,et al.  Covert and Side Channels Due to Processor Architecture , 2006, 2006 22nd Annual Computer Security Applications Conference (ACSAC'06).

[26]  Aurélien Francillon,et al.  C5: Cross-Cores Cache Covert Channel , 2015, DIMVA.

[27]  Nan Jiang,et al.  A detailed and flexible cycle-accurate Network-on-Chip simulator , 2013, 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[28]  Ruby B. Lee,et al.  A novel cache architecture with enhanced performance and security , 2008, 2008 41st IEEE/ACM International Symposium on Microarchitecture.

[29]  Rami G. Melhem,et al.  Simultaneous Multikernel GPU: Multi-tasking throughput processors via fine-grained sharing , 2016, 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[30]  Timothy G. Rogers,et al.  Locality-Centric Data and Threadblock Management for Massive GPUs , 2020, 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[31]  Hovav Shacham,et al.  Hey, you, get off of my cloud: exploring information leakage in third-party compute clouds , 2009, CCS.

[32]  David Kaeli,et al.  Exploiting Bank Conflict-based Side-channel Timing Leakage of GPUs , 2019, ACM Trans. Archit. Code Optim..

[33]  Nam Sung Kim,et al.  The case for GPGPU spatial multitasking , 2012, IEEE International Symposium on High-Performance Comp Architecture.

[34]  Avinash Karanth Kodi,et al.  Securing NoCs Against Timing Attacks with Non-Interference Based Adaptive Routing , 2018, 2018 Twelfth IEEE/ACM International Symposium on Networks-on-Chip (NOCS).

[35]  Nael B. Abu-Ghazaleh,et al.  Rendered Insecure: GPU Side Channel Attacks are Practical , 2018, CCS.

[36]  David Kaeli,et al.  Hardware/Software Obfuscation against Timing Side-channel Attack on a GPU , 2020, 2020 IEEE International Symposium on Hardware Oriented Security and Trust (HOST).

[37]  Danfeng Zhang,et al.  RCoal: Mitigating GPU Timing Attack via Subwarp-Based Randomized Coalescing Techniques , 2018, 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[38]  Alexandros G. Dimakis,et al.  Understanding contention-based channels and using them for defense , 2015, 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA).

[39]  Won Woo Ro,et al.  Warped-Slicer: Efficient Intra-SM Slicing through Dynamic Resource Partitioning for GPU Multiprogramming , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[40]  Natalie D. Enright Jerger,et al.  On-Chip Networks , 2009, On-Chip Networks.

[41]  Gernot Heiser,et al.  Last-Level Cache Side-Channel Attacks are Practical , 2015, 2015 IEEE Symposium on Security and Privacy.

[42]  Nael B. Abu-Ghazaleh,et al.  Constructing and Characterizing Covert Channels on GPGPUs , 2017, 2017 50th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[43]  Stefan Mangard,et al.  DRAMA: Exploiting DRAM Addressing for Cross-CPU Attacks , 2015, USENIX Security Symposium.

[44]  G. Edward Suh,et al.  Efficient Timing Channel Protection for On-Chip Networks , 2012, 2012 IEEE/ACM Sixth International Symposium on Networks-on-Chip.

[45]  Josep Torrellas,et al.  ReplayConfusion: Detecting cache-based covert channel attacks using record and replay , 2016, 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[46]  David R. Kaeli,et al.  Trident: A Hybrid Correlation-Collision GPU Cache Timing Attack for AES Key Recovery , 2021, 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA).

[47]  John Kim,et al.  Improving GPGPU resource utilization through alternative thread block scheduling , 2014, 2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA).

[48]  Guru Venkataramani,et al.  CC-Hunter: Uncovering Covert Timing Channels on Shared Processor Hardware , 2014, 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture.

[49]  William J. Dally,et al.  GPUs and the Future of Parallel Computing , 2011, IEEE Micro.

[50]  Adwait Jog,et al.  BCoal: Bucketing-Based Memory Coalescing for Efficient and Secure GPUs , 2020, 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[51]  Prabhat Kumar,et al.  Exploring concentration and channel slicing in on-chip network router , 2009, 2009 3rd ACM/IEEE International Symposium on Networks-on-Chip.

[52]  Klaus Wagner,et al.  Flush+Flush: A Fast and Stealthy Cache Attack , 2015, DIMVA.

[53]  John Kim,et al.  A Novel Covert Channel Attack Using Memory Encryption Engine Cache , 2019, 2019 56th ACM/IEEE Design Automation Conference (DAC).

[54]  Zhenyu Wu,et al.  Whispers in the Hyper-space: High-speed Covert Channel Attacks in the Cloud , 2012, USENIX Security Symposium.

[55]  Tor M. Aamodt,et al.  Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling , 2020, 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA).

[56]  Nael B. Abu-Ghazaleh,et al.  GPUGuard: mitigating contention based side and covert channel attacks on GPUs , 2019, ICS.

[57]  David R. Kaeli,et al.  Side-channel power analysis of a GPU AES implementation , 2015, 2015 33rd IEEE International Conference on Computer Design (ICCD).