Graph Support and Scheduling for OpenCL on Heterogeneous Multi-core Systems

Computation on heterogeneous multi-core systems has great opportunities for optimization which may include the compute resource scheduling such as workload distribution between CPU and GPU, as well as finding the best combination of tasks and compute devices for best performance. Currently, OpenCL, the parallel programming standard for heterogeneous computing, contains mainly low-level APIs to interact with the runtime and hardware device of each individual vendor. To apply efficient scheduling algorithm, the overall execution flow and information of OpenCL kernels must be considered. In this paper, we proposed computational graph support for OpenCL. The framework features computational graphs that store meta-data and execution dependencies of kernels. We then provide scheduling framework for OpenCL programs based on the graph information. In our optimization framework, the kernel task scheduling is based on the graph model. In addition, we have kernel code analysis for target device decision as well as runtime work-group size optimization. The preliminary experimental results show that our scheme enables significant performance enhancement, achieving about 1.59 times speedup relative to our neural network program baseline.