Cache Balancer: A communication latency and utilization aware resource manager

The increasing number of processors in today's many-core architectures has lead to new issues regarding memory management. The performance of many-core processors is often limited by the communication latency incurred in data transfers between different cores. Conventional memory allocators do not take such communication costs into account while allocating memory for application tasks at runtime. While a number of existing proposals address this issue, they result in the non-uniform utilization of available system resources. This work introduces Cache Balancer, a technique for dynamic memory allocation that addresses the limitations of state-of-the-art schemes. Cache Balancer introduces the access rate metric to measure the utilization of different cache banks in the system, and uses this at runtime to determine where memory is allocated. The technique reduces memory access latency by up to 63.4% by avoiding allocation of memory in over-utilized cache banks. Furthermore, Cache Balancer incorporates a runtime task mapper that utilizes information on the execution characteristics of tasks and the structure of the system interconnect in determining a mapping solution that results in optimal memory throughput. This results in additional memory access latency reductions of up to 14.5%, and combined execution time improvements of up to 22% as compared to state-of-the-art schemes.