Edge Computing-Enabled Deep Learning for Real-time Video Optimization in IIoT

Real-time multimedia applications have gained immense popularity in the industrial Internet of Things (IIoT) paradigm. Due to the impact of the complex industrial environment, the transmission of video streaming is usually unstable. In the duration of a low bandwidth transmission, existing optimization methods often reduce the original resolution of some frames in a random way to avoid the video interruption. If the key frames with some important content are selected to be transmitted with a low resolution, it will greatly reduce the effect of industrial supervision. In view of this challenge, a real-time video streaming optimization method by reducing the number of video frames transmitted in the IIoT environment is proposed. Concretely, a deep learning-based object detection algorithm is recruited to effectively select the key frames in our method. The key frames with the original resolution will be transmitted along with audio data. As some nonkey frames are selectively discarded, it is helpful for smooth network transmitting with fewer bandwidth requirements. Moreover, we employ edge servers to run the object detection algorithm, and adjust video transmission flexibly. Extensive experiments are conducted to validate the effectiveness, and dependability of our method.

[1]  M. Siekkinen,et al.  Edge Computing Assisted Adaptive Mobile Video Streaming , 2019, IEEE Transactions on Mobile Computing.

[2]  Victor C. M. Leung,et al.  Decentralized Resource Allocation for Video Transcoding and Delivery in Blockchain-Based System With Mobile Edge Computing , 2019, IEEE Transactions on Vehicular Technology.

[3]  Ahmad Khonsari,et al.  Cost-Effective Low-Delay Design for Multiparty Cloud Video Conferencing , 2017, IEEE Transactions on Multimedia.

[4]  Tao Jiang,et al.  Edge Computing Framework for Cooperative Video Processing in Multimedia IoT Systems , 2018, IEEE Transactions on Multimedia.

[5]  Sung Wook Baik,et al.  Intelligent Embedded Vision for Summarization of Multiview Videos in IIoT , 2020, IEEE Transactions on Industrial Informatics.

[6]  Nuno Vasconcelos,et al.  Cascade R-CNN: Delving Into High Quality Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Maria A. Amer,et al.  Effective keyframe extraction from RGB and RGB-D video sequences , 2017, 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA).

[8]  Xi Wang,et al.  Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification , 2015, ACM Multimedia.

[9]  Mohammad R. Khosravi,et al.  Efficient payload communications for IoT-enabled ViSAR vehicles using discrete cosine transform-based quasi-sparse bit injection , 2019, EURASIP J. Wirel. Commun. Netw..

[10]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[12]  Miguel A. Martínez-del-Amor,et al.  Movies tags extraction using deep learning , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[13]  Ming Ma,et al.  Effective moving object detection in H.264/AVC compressed domain for video surveillance , 2019, Multimedia Tools and Applications.

[14]  Yantian Hou,et al.  Edge-Assisted CNN Inference over Encrypted Data for Internet of Things , 2019, SecureComm.

[15]  Zhang Jia-cheng,et al.  Fast and high sensitivity focusing evaluation function , 2010 .

[16]  Sadegh Samadi,et al.  Reliable Data Aggregation in Internet of ViSAR Vehicles Using Chained Dual-Phase Adaptive Interpolation and Data Embedding , 2020, IEEE Internet of Things Journal.

[17]  Tammam Tillo,et al.  Optical Flow Estimation with Deep Learning, a Survey on Recent Advances , 2020, Deep Biometrics.

[18]  Rabab Kreidieh Ward,et al.  Deep learning for pixel-level image fusion: Recent advances and future prospects , 2018, Inf. Fusion.

[19]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[20]  Hongbo Jiang,et al.  Adaptive Wireless Video Streaming Based on Edge Computing: Opportunities and Approaches , 2019, IEEE Transactions on Services Computing.

[21]  Yujie Li,et al.  Key frame extraction from first-person video with multi-sensor integration , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[22]  Houbing Song,et al.  Security of the Internet of Things: Vulnerabilities, Attacks, and Countermeasures , 2019, IEEE Communications Surveys & Tutorials.

[23]  Victor C. M. Leung,et al.  Distributed Resource Allocation in Blockchain-Based Video Streaming Systems With Mobile Edge Computing , 2019, IEEE Transactions on Wireless Communications.