AdaEnlight

The ubiquity of camera-embedded devices and the advances in deep learning have stimulated various intelligent mobile video applications. These applications often demand on-device processing of video streams to deliver real-time, high-quality services for privacy and robustness concerns. However, the performance of these applications is constrained by the raw video streams, which tend to be taken with small-aperture cameras of ubiquitous mobile platforms in dim light. Despite extensive low-light video enhancement solutions, they are unfit for deployment to mobile devices due to their complex models and and ignorance of system dynamics like energy budgets. In this paper, we propose AdaEnlight, an energy-aware low-light video stream enhancement system on mobile devices. It achieves real-time video enhancement with competitive visual quality while allowing runtime behavior adaptation to the platform-imposed dynamic energy budgets. We report extensive experiments on diverse datasets, scenarios, and platforms and demonstrate the superiority of AdaEnlight compared with state-of-the-art low-light image and video enhancement solutions.

[1]  L. Kong,et al.  Wi-Fruit , 2021, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[2]  Yunxin Liu,et al.  nn-Meter: towards accurate latency prediction of deep-learning model inference on diverse edge devices , 2021, MobiSys.

[3]  Ibraheem Kasim Ibraheem,et al.  Embedded Design and Implementation of Mobile Robot for Surveillance Applications , 2021 .

[4]  Chen Change Loy,et al.  Low-Light Image and Video Enhancement Using Deep Learning: A Survey , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Chen Change Loy,et al.  Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Seongwon Kim,et al.  Supremo: Cloud-Assisted Low-Latency Super-Resolution in Mobile Devices , 2020, IEEE Transactions on Mobile Computing.

[7]  Hoi-Jun Yoo,et al.  The Hardware and Algorithm Co-Design for Energy-Efficient DNN Processor on Edge/Mobile Devices , 2020, IEEE Transactions on Circuits and Systems I: Regular Papers.

[8]  V. Padmanabhan,et al.  InSight , 2020, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies.

[9]  Dongsu Han,et al.  Neural-Enhanced Live Streaming: Improving Live Video Ingest via Online Learning , 2020, SIGCOMM.

[10]  Jasper S. Wijnands,et al.  Real-time monitoring of driver drowsiness on mobile platforms using 3D neural networks , 2019, Neural Computing and Applications.

[11]  Nicholas D. Lane,et al.  MobiSR: Efficient On-Device Super-Resolution through Heterogeneous Mobile Processors , 2019, MobiCom.

[12]  Lili Qiu,et al.  Jigsaw: Robust Live 4K Video Streaming , 2019, MobiCom.

[13]  Xin He,et al.  NNest: Early-Stage Design Space Exploration Tool for Neural Network Inference Accelerators , 2018, ISLPED.

[14]  Hui Liu,et al.  On-Demand Deep Model Compression for Mobile Devices: A Usage-Driven Model Selection Framework , 2018, MobiSys.

[15]  Dongsu Han,et al.  How will Deep Learning Change Internet Video Delivery? , 2017, HotNets.

[16]  M. Abdullah-Al-Wadud,et al.  An adaptive gamma correction for image enhancement , 2016, EURASIP J. Image Video Process..

[17]  Yi Sun,et al.  CS2P: Improving Video Bitrate Selection and Adaptation with Data-Driven Throughput Prediction , 2016, SIGCOMM.

[18]  Soumik Sarkar,et al.  LLNet: A deep autoencoder approach to natural low-light image enhancement , 2015, Pattern Recognit..

[19]  Luca Benini,et al.  System-level power optimization: techniques and tools , 1999, Proceedings. 1999 International Symposium on Low Power Electronics and Design (Cat. No.99TH8477).

[20]  Jason Flinn,et al.  Energy-aware adaptation for mobile applications , 1999, SOSP.

[21]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..