HetroCV: Auto-tuning Framework and Runtime for Image Processing and Computer Vision Applications on Heterogeneous Platform

With the wide adoption of high-performance processors and accelerators, large-scale computer vision applications have gained great performance improvement. However, it often requires extensive experiments and expertise to achieve optimal performance from manually-tuned programs, and the programs often need to be re-tuned when transplanted to a different platform, or using a different system configuration. To overcome this problem, in this paper we proposed Hetro CV, a programmer-directed auto-tuning framework and runtime for computer vision applications on heterogeneous CPU-MIC platform. In Hetro CV auto-tuning framework, computation units in the application pipeline are categorized in to one of three patterns: Map, Stencil and MapReduce, and program statistics are extracted from units' meta-information. Machine learning is adopted to train models for each pattern using the tuned parameters and program statistics from trial run sets, so that when a new unit is presented, Hetro CV auto tuner can use the corresponding trained model to generate optimized tuning parameters. In Hetro CV runtime, performance models for processor and co-processor are built to predict the prospective execution time of each computation unit in the application pipeline. We adopted the maximum-throughput mapping strategy, thus each unit would be mapped dynamically to the processor/co-processor queue, which would generate the minimum overall execution time. Experiments on two medical image processing applications running on heterogeneous platform composed of Intel Xeon CPU and Intel Phi co-processor showed advanced performance over naive Open MP tuning and Genetic Algorithm (GA) based heuristic tuning.