The Execution Migration Machine

On-chip interconnect power already forms a significant portion of the power consumed by chip multiprocessors (CMPs), and with continued transistor scaling leading to higher and higher core counts, chip power will be increasingly dominated by the on-chip network. For massive multicores to be feasible, therefore, it will be necessary to significantly reduce total on-chip data movement. Since most of the traffic is related to bringing data to the locus of computation, one solution is to enable threads to efficiently migrate across the chip and execute near the data they access. In this paper, we present the detailed implementation of hardware-level instruction-granularity thread migration in a 110-core CMP. Implemented in 45nm ASIC technology, the chip occupies 100mm2 and is currently in the fabrication stage. With a custom stack-based ISA to enable partial context migration, when there is no network congestion, our implementation provides end-to-end migration latency of 4 cycles between neighboring cores with a minimum thread context, and 33 cycles between the farthest cores with a maximum context. To supplement a remote-cache-access-based shared memory paradigm, our cores learn a thread’s data access patterns and migrate threads automatically. Through RTL-level simulation, we demonstrate that migration can reduce on-chip data movement by up to 14× at a relatively small area cost of 23%.