Memory Aware Thread Aggregation Framework for Dynamic Parallelism in GPUs

Problems using matrices and vectors exhibit regular data-level parallelism, which yields maximum throughput when executed in GPUs. But problems which use data in the form of hierarchical data structures such as adaptive grids, sparse graphs and trees, and applications using recursion, may not have regular parallelism. The parallelism in algorithms based on these data structures are discovered dynamically as they are nested. Depending on the data being processed, each task may need to perform a variable amount of computation. The amount of parallelism available at any point of execution cannot be determined apriori as it is data dependent and evolves with the computation itself. This non-determinism is due to the existence of control flow divergence, diverse memory access patterns and data dependent parallelism in the problems, which do not fit into GPU architectures as the natural choice and result in much lesser utilization of GPU resources. A three-level thread aggregation is proposed in this paper for enhancing the resource utilization and occupancy. Threads are executed in the appropriate level after aggregation into larger grids. Programs from the Rodinia benchmark and NVIDIA SDK are used to test the framework. The occupancy improves from 2X to 15X and the performance improvement up to 50% in some cases.