CMH: compression management for improving capacity in the hybrid memory cube

The Hybrid Memory Cube (HMC) is a novel 3D memory architecture that efficiently improves bandwidth and saves energy. However, due to limitations in scalability and power density of a DRAM bit cell, the physical data capacity of an individual HMC is relatively modest and unlikely to grow significantly and it is likely to be a challenge in adopting the HMC for big data in high-performance computing. In this paper, we propose a new strategy to increase the effective data capacity of the HMC, called Compression Management for HMC (CMH). CMH is incorporated in the logic layer of the HMC. By selectively compressing data during transmission and storing the selectively compressed data in the 3D memory stack, CMH increases data capacity while also improving effective bandwidth. For several memory-intensive benchmarks, our results show that CMH reduces pressure on memory capacity by 64.4%, and improves bandwidth by 42.4%. Similarly good results are observed for multi-programmed workloads, reducing capacity 66.2% and improving bandwidth 47.8%. Although compression has latency overhead, by introducing a small cache in the HMC logic layer to store metadata for compression, CMH mitigates any increase in transaction latency. The overhead in instructions per cycle is a minimal 1.2% and 1.5%, respectively, for single-core and multi-core workloads. The IPC is stable and is not harmed by the inclusion of compression.