Accelerating NNEF Framework on OpenCL Devices Using clDNN

In recent years, many neural network frameworks have been proposed, such as TensorFlow, Caffe, Keras, and PyTorch. These frameworks have their file formats to store trained parameters, but these file formats are not compatible with each other. Therefore, Khronos has proposed a Neural Network Exchange Format (NNEF) that allows engineers to convert a pre-trained model across different frameworks. However, the NNEF is a file format of intermediate representation (IR). Hence, an efficient solution to execute the inference of the AI model through the NNEF on different devices deserves further research. In this paper, We propose a translator to convert the NNEF to clDNN. The clDNN is an open-source inference SDK, which is provided high-performance computation APIs on Intel hardware platforms. In our preliminary experiment results, the translator can speed up six times compared to the C implementation for the execution time of MobileNet_v1.