Industrial Pervasive Edge Computing-Based Intelligence IoT for Surveillance Saliency Detection

Numerous surveillance data processing is crucial in the internet-of-things systems with pervasive edge computing. In this processing, salient object detection from surveillance videos plays an important role because it provides the human-concerned semantic cue for various industrial tasks. However, it is still challenging for the existing studies with two aspects. The first one is the redundant saliency information from moving background to disturb the detection of salient objects. The second one is the difficulty to model the spatiotemporal saliency uncertainty. To overcome these challenges, an intelligent approach is proposed for surveillance saliency detection. It enables a region-proposal-based optical flow strategy to suppress the saliency enhancement of non-salient regions due to the moving background. Besides, it develops the bidirectional Bayesian state transition strategy to model the motion uncertainty for refining the spatiotemporal saliency feature. Extensive experiments have been performed on two datasets (the increase of ${F_{\beta}}$ is larger than 0.01 for DAVIS, and larger than 0.015 for UVSD), and the comparison with seven methods to evaluate the effectiveness of the proposed approach.

[1]  Ramesh Raskar,et al.  Learning Gaze Transitions from Depth to Improve Video Saliency Estimation , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[2]  Richard P. Wildes,et al.  Dynamically encoded actions based on spacetime saliency , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Andrey Somov,et al.  Pervasive Agriculture: IoT-Enabled Greenhouse for Plant Growth Control , 2018, IEEE Pervasive Computing.

[4]  Xin Wang,et al.  Salient Object Detection in the Distributed Cloud-Edge Intelligent Network , 2020, IEEE Network.

[5]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  J. Wolfe,et al.  What attributes guide the deployment of visual attention and how do they do it? , 2004, Nature Reviews Neuroscience.

[7]  Huchuan Lu,et al.  Saliency detection via Cellular Automata , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Shinichiro Omachi,et al.  Object-Based Video Coding by Visual Saliency and Temporal Correlation , 2020, IEEE Transactions on Emerging Topics in Computing.

[9]  Moncef Gabbouj,et al.  Spatiotemporal Saliency Estimation by Spectral Foreground Detection , 2018, IEEE Transactions on Multimedia.

[10]  Guang Yang,et al.  SaliencyGAN: Deep Learning Semisupervised Salient Object Detection in the Fog of IoT , 2020, IEEE Transactions on Industrial Informatics.

[11]  Wonjun Kim,et al.  Spatiotemporal Saliency Detection Using Textural Contrast and Its Applications , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  James J. Clark,et al.  Going from Image to Video Saliency: Augmenting Image Salience with Dynamic Attentional Push , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Sung Wook Baik,et al.  Intelligent Embedded Vision for Summarization of Multiview Videos in IIoT , 2020, IEEE Transactions on Industrial Informatics.

[14]  Jia Li,et al.  Deep3DSaliency: Deep Stereoscopic Video Saliency Detection Model by 3D Convolutional Networks , 2019, IEEE Transactions on Image Processing.

[15]  Ghassan AlRegib,et al.  Unsupervised Uncertainty Estimation Using Spatiotemporal Cues in Video Saliency Detection , 2018, IEEE Transactions on Image Processing.

[16]  Zhuowen Tu,et al.  Deeply Supervised Salient Object Detection with Short Connections , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Yuhui Zheng,et al.  Weighted LIC-Based Structure Tensor With Application to Image Content Perception and Processing , 2021, IEEE Transactions on Industrial Informatics.

[18]  Chang-Su Kim,et al.  Spatiotemporal Saliency Detection for Video Sequences Based on Random Walk With Restart , 2015, IEEE Transactions on Image Processing.

[19]  Linwei Ye,et al.  Saliency Detection for Unconstrained Videos Using Superpixel-Level Graph and Spatiotemporal Propagation , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Irfan Mehmood,et al.  Edge Intelligence-Assisted Smoke Detection in Foggy Surveillance Environments , 2020, IEEE Transactions on Industrial Informatics.

[21]  Jun Fang,et al.  A Saliency-Based Method for Early Smoke Detection in Video Sequences , 2016 .

[22]  Qi Zhao,et al.  Attentive Systems: A Survey , 2017, International Journal of Computer Vision.

[23]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Li Xu,et al.  Hierarchical Image Saliency Detection on Extended CSSD , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Khan Muhammad,et al.  Cost-Effective Video Summarization Using Deep CNN With Hierarchical Weighted Fusion for IoT Surveillance Networks , 2020, IEEE Internet of Things Journal.

[26]  Jie Yang,et al.  Saliency Detection by Fully Learning a Continuous Conditional Random Field , 2017, IEEE Transactions on Multimedia.

[27]  Qinghua Hu,et al.  SG-FCN: A Motion and Memory-Based Deep Learning Model for Video Saliency Detection , 2018, IEEE Transactions on Cybernetics.

[28]  Qi Zhao,et al.  Learning saliency-based visual attention: A review , 2013, Signal Process..

[29]  Jingbo Chen,et al.  Spatiotemporal Saliency Detection Based on Maximum Consistency Superpixels Merging for Video Analysis , 2020, IEEE Transactions on Industrial Informatics.

[30]  Luc Van Gool,et al.  A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Yu Hen Hu,et al.  Video Saliency Detection via Graph Clustering With Motion Energy and Spatiotemporal Objectness , 2019, IEEE Transactions on Multimedia.

[32]  Dewen Hu,et al.  Salient Region Detection via Integrating Diffusion-Based Compactness and Local Contrast , 2015, IEEE Transactions on Image Processing.

[33]  Jaime Lloret,et al.  Robust Image Hashing Based Efficient Authentication for Smart Industrial Environment , 2019, IEEE Transactions on Industrial Informatics.

[34]  Wonjun Kim,et al.  Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Christina J. Howard,et al.  Unexpected changes in direction of motion attract attention. , 2010, Attention, perception & psychophysics.

[36]  Sabine Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Yue Cao,et al.  A Pervasive Integration Platform of Low-Cost MEMS Sensors and Wireless Signals for Indoor Localization , 2018, IEEE Internet of Things Journal.

[38]  Nuno Vasconcelos,et al.  Spatiotemporal Saliency in Dynamic Scenes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Ali Borji,et al.  Revisiting Video Saliency: A Large-Scale Benchmark and a New Model , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[40]  Dong Xu,et al.  Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey , 2018, IEEE Signal Processing Magazine.

[41]  Yoshua Bengio,et al.  A Recurrent Latent Variable Model for Sequential Data , 2015, NIPS.

[42]  David Dagan Feng,et al.  Robust saliency detection via regularized random walks ranking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Thomas Mauthner,et al.  Encoding based saliency detection for videos and images , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Shuo Li,et al.  Trustful Internet of Surveillance Things Based on Deeply Represented Visual Co-Saliency Detection , 2020, IEEE Internet of Things Journal.

[45]  Yuan Xie,et al.  Flow Guided Recurrent Neural Encoder for Video Salient Object Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[46]  Vinod Vokkarane,et al.  A New Deep Learning-Based Food Recognition System for Dietary Assessment on An Edge Computing Service Infrastructure , 2018, IEEE Transactions on Services Computing.

[47]  Bing Li,et al.  Salient Object Detection via Structured Matrix Decomposition. , 2016, IEEE transactions on pattern analysis and machine intelligence.

[48]  Ling Shao,et al.  Video Salient Object Detection via Fully Convolutional Networks , 2017, IEEE Transactions on Image Processing.

[49]  Sung Wook Baik,et al.  Secure Surveillance Framework for IoT Systems Using Probabilistic Image Encryption , 2018, IEEE Transactions on Industrial Informatics.

[50]  Saumik Bhattacharya,et al.  Visual Saliency Detection Using Spatiotemporal Decomposition , 2018, IEEE Transactions on Image Processing.

[51]  Mei Li,et al.  Multimodal GAN for Energy Efficiency and Cloud Classification in Internet of Things , 2019, IEEE Internet of Things Journal.

[52]  Yizhou Yu,et al.  Deep Contrast Learning for Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Qingming Huang,et al.  Image Saliency Detection Video Saliency Detection Co-saliency Detection Temporal RGBD Saliency Detection Motion , 2018 .

[54]  Khan Muhammad,et al.  DeepReS: A Deep Learning-Based Video Summarization Strategy for Resource-Constrained Industrial Surveillance Scenarios , 2020, IEEE Transactions on Industrial Informatics.