SecDeep: Secure and Performant On-device Deep Learning Inference Framework for Mobile and IoT Devices

There is an increasing emphasis on securing deep learning (DL) inference pipelines for mobile and IoT applications with privacy-sensitive data. Prior works have shown that privacy-sensitive data can be secured throughout deep learning inferences on cloud-offloaded models through trusted execution environments such as Intel SGX. However, prior solutions do not address the fundamental challenges of securing the resource-intensive inference tasks on low-power, low-memory devices (e.g., mobile and IoT devices), while achieving high performance. To tackle these challenges, we propose SecDeep, a low-power DL inference framework demonstrating that both security and performance of deep learning inference on edge devices are well within our reach. Leveraging TEEs with limited resources, SecDeep guarantees full confidentiality for input and intermediate data, as well as the integrity of the deep learning model and framework. By enabling and securing neural accelerators, SecDeep is the first of its kind to provide trusted and performant DL model inferencing on IoT and mobile devices. We implement and validate SecDeep by interfacing the ARM NN DL framework with ARM TrustZone. Our evaluation shows that we can securely run inference tasks with 16× to 172× faster performance than no acceleration approaches by leveraging edge-available accelerators.

[1]  Tommaso Frassetto,et al.  Offline Model Guard: Secure and Private ML on Mobile Devices , 2020, 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[2]  Lin Zhong,et al.  Ginseng: Keeping Secrets in Registers When You Distrust the Operating System , 2019, NDSS.

[3]  Nael B. Abu-Ghazaleh,et al.  SafeSpec: Banishing the Spectre of a Meltdown with Leakage-Free Speculation , 2018, 2019 56th ACM/IEEE Design Automation Conference (DAC).

[4]  Mihir Bellare,et al.  Format-Preserving Encryption , 2009, IACR Cryptol. ePrint Arch..

[5]  Shouhuai Xu,et al.  TEE: a virtual DRTM based execution environment for secure cloud-end computing , 2010, CCS '10.

[6]  Luis Ceze,et al.  Neural Acceleration for General-Purpose Approximate Programs , 2014, IEEE Micro.

[7]  Song Han,et al.  AMC: AutoML for Model Compression and Acceleration on Mobile Devices , 2018, ECCV.

[8]  Dan Boneh,et al.  Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware , 2018, ICLR.

[9]  Dimitrios Pendarakis,et al.  Securing Input Data of Deep Learning Inference Systems via Partitioned Enclave Execution , 2018, ArXiv.

[10]  Nael B. Abu-Ghazaleh,et al.  Spectre Returns! Speculation Attacks Using the Return Stack Buffer , 2018, IEEE Design & Test.

[11]  Wolfgang Schröder-Preikschat,et al.  Automatic OS Kernel TCB Reduction by Leveraging Compile-Time Configurability , 2012, HotDep.

[12]  Carole-Jean Wu,et al.  Machine Learning at Facebook: Understanding Inference at the Edge , 2019, 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[13]  Shweta Shinde,et al.  Privado: Practical and Secure DNN Inference , 2018, ArXiv.

[14]  Minsu Kim,et al.  ATRA: Address Translation Redirection Attack against Hardware-based External Monitors , 2014, CCS.

[15]  Mani B. Srivastava,et al.  PROTC: PROTeCting Drone's Peripherals through ARM TrustZone , 2017, DroNet@MobiSys.

[16]  Quan Chen,et al.  Hypervision Across Worlds: Real-time Kernel Protection from the ARM TrustZone Secure World , 2014, CCS.

[17]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[18]  Mianxiong Dong,et al.  Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing , 2018, IEEE Network.

[19]  Deian Stefan,et al.  CT-wasm: type-driven secure cryptography for the web ecosystem , 2018, Proc. ACM Program. Lang..

[20]  Rüdiger Kapitza,et al.  Telling Your Secrets without Page Faults: Stealthy Page Table-Based Attacks on Enclaved Execution , 2017, USENIX Security Symposium.

[21]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[23]  Zhiyuan Zheng,et al.  Towards Improving Data Validity of Cyber-Physical Systems through Path Redundancy , 2017, CPSS@AsiaCCS.

[24]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Valerio Schiavoni,et al.  On The Performance of ARM TrustZone , 2019, ArXiv.

[26]  Xu Chen,et al.  Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy , 2018, MECOMM@SIGCOMM.

[27]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[28]  Miao Yu,et al.  Trusted Display on Untrusted Commodity Platforms , 2015, CCS.

[29]  William A. Arbaugh,et al.  A secure and reliable bootstrap architecture , 1997, Proceedings. 1997 IEEE Symposium on Security and Privacy (Cat. No.97CB36097).

[30]  Trent Jaeger,et al.  TrustShadow: Secure Execution of Unmodified Applications with ARM TrustZone , 2017, MobiSys.

[31]  Rodrigo Bruno,et al.  Graviton: Trusted Execution Environments on GPUs , 2018, OSDI.

[32]  Sebastian Nowozin,et al.  Oblivious Multi-Party Machine Learning on Trusted Processors , 2016, USENIX Security Symposium.

[33]  Jason Cong,et al.  Scaling for edge inference of deep neural networks , 2018 .

[34]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[35]  Mani Srivastava,et al.  VirtSense: Virtualize Sensing through ARM TrustZone on Internet-of-Things , 2018 .

[36]  Samuel S. Ogden,et al.  MODI: Mobile Deep Inference Made Efficient by Edge Computing , 2018, HotEdge.

[37]  Kaibin Huang,et al.  Towards an Intelligent Edge: Wireless Communication Meets Machine Learning , 2018, ArXiv.

[38]  Fengyuan Xu,et al.  Occlumency: Privacy-preserving Remote Deep-learning Inference Using SGX , 2019, MobiCom.