Adaptive Tiling: Applying Fixed-size Systolic Arrays To Sparse Convolutional Neural Networks

We introduce adaptive tiling, a method of partitioning layers in a sparse convolutional neural network (CNN) into blocks of filters and channels, called tiles, each implementable with a fixed-size systolic array. By allowing a tile to adapt its size so that it can cover a large sparse area, we minimize the total number of tiles, or equivalently, the number of systolic array calls required to perform CNN inference. The proposed scheme resolves a challenge of applying systolic array architectures, traditionally designed for dense matrices, to sparse CNNs. To validate the approach, we construct a highly sparse Lasso-Mobile network by pruning MobileNet trained with an $\ell_{1}$ regularization penalty, and demonstrate that adaptive tiling can lead to a 2- $3\mathbf{x}$ reduction in systolic array calls, on Lasso-Mobile, for several benchmark datasets.