Towards Optimal Winograd Convolution on Manycores

As convolutional layers are computationally intensive and dominate the total execution time of modern, deep Convolutional neural networks (ConvNets) [15, 22, 24, 25], many efforts have been made to improve the performance of the convolutional primitives for CPUs [1, 6, 26, 31, 33], GPUs [4, 7, 19, 27] or both [32]. An important class of improvements is to reduce the computations required for a convolution. Several efforts employed FFT-based convolutions to reduce the required computations for GPUs [19, 27] and CPUs [31, 32]. Recently, Lavin et al. [16] proposed an algorithm based on the Winograd algorithm for minimal filtering, originally developed for fast computation of finite impulse response (FIR) filters [30]. The key idea of the Winograd–based convolution is similar to the one based on FFTs. The inputs and the kernels are first transformed, and then an element–wise multiplication, which is an equivalent problem to a matrix multiplication, is performed. An inverse transformation of the result yields the result of the convolution. Unlike the FFT–based convolution, where the point-wise multiplications are done in the complex domain, the Winograd–based convolution operates on real numbers, thus requiring fewer operations. After Lavin et al. [16] demonstrated that Winograd–based convolution can be more efficient than FFT in reducing the number of multiplications, especially for small 2D kernels (e.g. 3 × 3), Nervana [4] and Nvidia’s cuDNN [7] had implemented Winograd–based convolution for GPUs. CPU implementations were also provided by FALCON [1], LIBXSMM[5], Intel MKL-DNN [2] and Budden et al. [6]. In addition, currently available implementations support only 2D convolutions and single kernel size (3 × 3), which restricts the range of the application of Winograd–based convolution. 3D ConvNets are becoming important, as they have been successfully applied to many fields [9, 14, 20, 21]. Our work on the design, implementation and evaluation of a fast Winograd-based implementation is motivated by the fact that current Winograd–based implementations for multicore or manycore CPUs perform well below the hardware capability. In many cases, they under–performed more computationally expensive, but more optimized implementations, such as direct convolutions.