Shifting Capsule Networks from the Cloud to the Deep Edge

Capsule networks (CapsNets) are an emerging trend in image processing. In contrast to a convolutional neural network, CapsNets are not vulnerable to object deformation, as the relative spatial information of the objects is preserved across the network. However, their complexity is mainly related with the capsule structure and the dynamic routing mechanism, which makes it almost unreasonable to deploy a CapsNet, in its original form, in a resource-constrained device powered by a small microcontroller (MCU). In an era where intelligence is rapidly shifting from the cloud to the edge, this high complexity imposes serious challenges to the adoption of CapsNets at the very edge. To tackle this issue, we present an API for the execution of quantized CapsNets in Cortex-M and RISC-V MCUs. Our software kernels extend the Arm CMSIS-NN and RISC-V PULP-NN, to support capsule operations with 8-bit integers as operands. Along with it, we propose a framework to perform post-training quantization of a CapsNet. Results show a reduction in memory footprint of almost 75%, with a maximum accuracy loss of 1%. In terms of throughput, our software kernels for the Arm Cortex-M are, at least, 5.70x faster than a pre-quantized CapsNet running on an NVIDIA GTX 980 Ti graphics card. For RISC-V, the throughout gain increases to 26.28x and 56.91x for a singleand octa-core configuration, respectively.

[1]  Xuecheng Zou,et al.  Efficient Hardware Architecture of Convolutional Neural Network for ECG Classification in Wearable Healthcare Device , 2021, IEEE Transactions on Circuits and Systems I: Regular Papers.

[2]  Neelavathy Pari S,et al.  Real-Time Traffic Sign Detection using Capsule Network , 2019, 2019 11th International Conference on Advanced Computing (ICoAC).

[3]  Weigong Zhang,et al.  Enabling Highly Efficient Capsule Networks Processing Through A PIM-Based Architecture Design , 2019, 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[4]  Vikas Chandra,et al.  CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs , 2018, ArXiv.

[5]  Deyi Xiong,et al.  Towards Linear Time Neural Machine Translation with Capsule Networks , 2018, EMNLP.

[6]  Daniel Oliveira,et al.  Detecting Driver’s Fatigue, Distraction and Activity Using a Non-Intrusive Ai-Based Monitoring System , 2019, J. Artif. Intell. Soft Comput. Res..

[7]  Kouichi Sakurai,et al.  One Pixel Attack for Fooling Deep Neural Networks , 2017, IEEE Transactions on Evolutionary Computation.

[8]  Talha Burak Alakus,et al.  Convolutional capsnet: A novel artificial neural network approach to detect COVID-19 disease from X-ray images using capsule networks , 2020, Chaos, Solitons & Fractals.

[9]  Luca Benini,et al.  GAP-8: A RISC-V SoC for AI at the Edge of the IoT , 2018, 2018 IEEE 29th International Conference on Application-specific Systems, Architectures and Processors (ASAP).

[10]  Miguel Costa,et al.  The Future of Low-End Motes in the Internet of Things: A Prospective Paper , 2020, Electronics.

[11]  Geoffrey E. Hinton,et al.  Transforming Auto-Encoders , 2011, ICANN.

[12]  Hoi-Jun Yoo,et al.  A 1.15 TOPS/W Energy-Efficient Capsule Network Accelerator for Real-Time 3D Point Cloud Segmentation in Mobile Environment , 2020, IEEE Transactions on Circuits and Systems II: Express Briefs.

[13]  Muhammad Shafique,et al.  DESCNet: Developing Efficient Scratchpad Memories for Capsule Network Hardware , 2020, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[14]  Luca Benini,et al.  PULP-NN: accelerating quantized neural networks on parallel ultra-low-power RISC-V processors , 2019, Philosophical Transactions of the Royal Society A.

[15]  Mianxiong Dong,et al.  Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing , 2018, IEEE Network.

[16]  Razvan C. Bunescu,et al.  Galaxy morphology prediction using capsule networks , 2018, Monthly Notices of the Royal Astronomical Society.

[17]  Taoufiq Gadi,et al.  From Auto-encoders to Capsule Networks: A Survey , 2021, E3S Web of Conferences.

[18]  Koosha Sadeghi,et al.  A System-Driven Taxonomy of Attacks and Defenses in Adversarial Machine Learning , 2020, IEEE Transactions on Emerging Topics in Computational Intelligence.

[19]  Bo Chen,et al.  Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Vincent Gripon,et al.  Quantization and Deployment of Deep Neural Networks on Microcontrollers , 2021, Sensors.

[21]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[22]  Sandro Pinto,et al.  Operating Systems for Internet of Things Low-End Devices: Analysis and Benchmarking , 2019, IEEE Internet of Things Journal.

[23]  C. John Glossner,et al.  Pruning and Quantization for Deep Neural Network Acceleration: A Survey , 2021, Neurocomputing.

[24]  Qiang Chen,et al.  Towards Accurate Post-training Network Quantization via Bit-Split and Stitching , 2020, ICML.

[25]  Randy H. Katz,et al.  A Berkeley View of Systems Challenges for AI , 2017, ArXiv.

[26]  Muhammad Shafique,et al.  Q-CapsNets: A Specialized Framework for Quantizing Capsule Networks , 2020, 2020 57th ACM/IEEE Design Automation Conference (DAC).

[27]  Muhammad Shafique,et al.  FEECA: Design Space Exploration for Low-Latency and Energy-Efficient Capsule Network Accelerators , 2021, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[28]  Adebayo Felix Adekoya,et al.  Capsule Networks - A survey , 2019, J. King Saud Univ. Comput. Inf. Sci..

[29]  Horácio C. Neto,et al.  Moving Deep Learning to the Edge , 2020, Algorithms.

[30]  Xin Fu,et al.  Enabling Highly Efficient Capsule Networks Processing Through Software-Hardware Co-Design , 2021, IEEE Transactions on Computers.

[31]  Kurt Keutzer,et al.  A Survey of Quantization Methods for Efficient Neural Network Inference , 2021, Low-Power Computer Vision.

[32]  Patrick Judd,et al.  Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation , 2020, ArXiv.

[33]  Yanzhi Wang,et al.  3D Capsule Networks for Object Classification With Weight Pruning , 2020, IEEE Access.

[34]  Geoffrey E. Hinton,et al.  Matrix capsules with EM routing , 2018, ICLR.

[35]  Amara Dinesh Kumar,et al.  Novel Deep Learning Model for Traffic Sign Detection Using Capsule Networks , 2018, ArXiv.