Exploiting Computation Reuse in Cloud-Based Deep Learning via Input Reordering

Recently, deep learning (DL) becomes increasingly important since its transformative effect on a wide range of applications. During inference process, the DL model is deployed on the cloud to answer online queries. One crucial issue in the progress of DL inference is energy consumption, which significantly retards computation performance. Therefore, many previous investigations decrease the energy consumption via computation reuse technique based on similarity. However, if input data consists individually from mobile devices, applying these schemes will significantly decline computation performance. Because in disordered individual inputs, similarity for reuse is difficult to exploit directly. Results of initial experimental observations show that (1) individual input data also has high similarity for reuse, and (2) the total similarity during computation process has a relation with the characteristics of input data. This motivates us to design a reordering scheme to enhance similarity for computation reuse. Our main approaches are using statistical theory to predict the similarities among input data, and determining the execution sequence. Based on these approaches, we propose an effective input reordering scheme for computation reuse to save energy consumption. The evaluation under various benchmarks demonstrates that the reordering scheme significantly outperforms the previous schemes, for instance, the computation reuse is enhanced to $1.1 \times$ and the energy consumption is minimized to 40% according to the configuration of traditional computation reuse technique.