Active-Routing: Compute on the Way for Near-Data Processing

With the explosion of data availability and fast data analytic requirements, data-intensive applications, characterized by their large memory footprint and low data reuse rate, are placing a significant amount of stress on modern memory systems and communication infrastructure. These workloads, ranging from data analytics to machine learning, exhibit considerable amount of aggregation operations over myriad of data, whose performance is limited due to the widening gap between high CPU compute density and deficient memory bandwidth. In this paper, we propose Active-Routing, an In-Network Compute Architecture enabling compute on the way for Near-Data Processing (NDP), which reduces the data movements across the memory hierarchy with a three-phase processing schedule. It first moves the computation to the data location and constructs an Active-Routing tree. Concurrently, Data Processing phase initiates when computations are offloaded to the memory network switches. Then, the network is enabled with compute capability to optimize the aggregation operations on the way along the dynamically built routing tree to reduce the traffic and parallelize computations in the memory network. Our evaluations show that Active-Routing can achieve up to 6x speedup with an average of 75% performance improvement across various benchmarks compared to a Baseline system integrated with a die-stacked memory network.