Toward efficient and intelligent video analytics with visual privacy protection for large-scale surveillance

Nowadays, the explosion of CCTV cameras has resulted in an increasing demand for distributed solutions to efficiently process the vast volume of video data. Otherwise, the use of surveillance when people are being watched remotely and recorded continuously has raised a significant threat to visual privacy. Using existing systems cannot prevent any party from exploiting unwanted personal data of others. In this paper, we develop an intelligent surveillance system with integrated privacy protection, where it is built on the top of big data tools, i.e., Kafka and Spark Streaming. To protect individual privacy, we propose a privacy-preserving solution based on effective face recognition and tracking mechanisms. Particularly, we associate body pose with face to reduce privacy leaks across video frames. The body pose is also exploited to infer person-centric information like human activities. Extensive experiments conducted on benchmark datasets further demonstrate the efficiency of our system for various vision tasks.

[1]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Ifeoma Ajunwa,et al.  Limitless Worker Surveillance , 2016 .

[3]  Lior Wolf,et al.  Live Face De-Identification in Video , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[4]  Weisong Shi,et al.  LAVEA: latency-aware video analytics on edge computing platform , 2017, SEC.

[5]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Ivan Sikiric,et al.  I Know That Person: Generative Full Body and Face De-identification of People in Images , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Janusz Konrad,et al.  Towards privacy-preserving recognition of human activities , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[8]  Chen Chen,et al.  Efficient and secure big data storage system with leakage resilience in cloud computing , 2018, Soft Comput..

[9]  Yong Jae Lee,et al.  Learning to Anonymize Faces for Privacy Preserving Action Detection , 2018, ECCV.

[10]  Bradley Malin,et al.  Preserving privacy by de-identifying face images , 2005, IEEE Transactions on Knowledge and Data Engineering.

[11]  Young-Gab Kim,et al.  Intelligent Video Data Security: A Survey and Open Challenges , 2021, IEEE Access.

[12]  Julianne H. Ayyad,et al.  Recognizing people from dynamic and static faces and bodies: Dissecting identity with a fusion approach , 2010, Vision Research.

[13]  Khan A. Wahid,et al.  Choice of Application Layer Protocols for Next Generation Video Surveillance Using Internet of Video Things , 2019, IEEE Access.

[14]  Rajiv Sabherwal,et al.  Mobile application security: Role of perceived privacy as the predictor of security perceptions , 2020, Int. J. Inf. Manag..

[15]  Mahadev Satyanarayanan,et al.  Scalable crowd-sourcing of video from mobile devices , 2013, MobiSys '13.

[16]  Bin Wu,et al.  Large Scale Video Data Analysis Based on Spark , 2015, 2015 International Conference on Cloud Computing and Big Data (CCBD).

[17]  Taghi M. Khoshgoftaar,et al.  A survey of open source tools for machine learning with big data in the Hadoop ecosystem , 2015, Journal of Big Data.

[18]  Weishan Zhang,et al.  A video cloud platform combing online and offline cloud computing technologies , 2015, Personal and Ubiquitous Computing.

[19]  Yutaka Ishibashi,et al.  IoT-based surveillance system for ubiquitous healthcare , 2016, IECON 2016 - 42nd Annual Conference of the IEEE Industrial Electronics Society.

[20]  Yongjie Chen,et al.  Security and Privacy of Smart Home Systems Based on the Internet of Things and Stereo Matching Algorithms , 2020, IEEE Internet of Things Journal.

[21]  Tal Hassner,et al.  Face recognition in unconstrained videos with matched background similarity , 2011, CVPR 2011.

[22]  Philip S. Yu,et al.  Distributed Deep Learning Model for Intelligent Video Surveillance Systems with Edge Computing , 2019, IEEE Transactions on Industrial Informatics.

[23]  Shree K. Nayar,et al.  Face swapping: automatically replacing faces in photographs , 2008, ACM Trans. Graph..

[24]  Young-Koo Lee,et al.  ML-HDP: A Hierarchical Bayesian Nonparametric Model for Recognizing Human Actions in Video , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Lidong Chen,et al.  An approach for fast and parallel video processing on Apache Hadoop clusters , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[26]  Toby Sharp,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR.

[27]  Ameet Talwalkar,et al.  MLlib: Machine Learning in Apache Spark , 2015, J. Mach. Learn. Res..

[28]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Ying Wu,et al.  Mining actionlet ensemble for action recognition with depth cameras , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Shree K. Nayar,et al.  Face swapping: automatically replacing faces in photographs , 2008, SIGGRAPH 2008.

[31]  Michael S. Ryoo,et al.  Privacy-Preserving Human Activity Recognition from Extreme Low Resolution , 2016, AAAI.

[32]  Michael J. Franklin,et al.  Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing , 2012, NSDI.

[33]  Luigi Cinque,et al.  2-D Skeleton-Based Action Recognition via Two-Branch Stacked LSTM-RNNs , 2020, IEEE Transactions on Multimedia.

[34]  Cristóbal Curio,et al.  Simple yet efficient real-time pose-based action recognition , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[35]  Luc Van Gool,et al.  Temporal Segment Networks: Towards Good Practices for Deep Action Recognition , 2016, ECCV.

[36]  Tribhuvanesh Orekondy,et al.  Connecting Pixels to Privacy and Utility: Automatic Redaction of Private Information in Images , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[37]  Xun Yi,et al.  Towards secure big data analytic for cloud-enabled applications with fully homomorphic encryption , 2020, J. Parallel Distributed Comput..

[38]  Tribhuvanesh Orekondy,et al.  Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[39]  Josep Domingo-Ferrer,et al.  Coprivacy: Towards a Theory of Sustainable Privacy , 2010, Privacy in Statistical Databases.

[40]  Sheridan Jeary,et al.  Re-identification attacks - A systematic literature review , 2016, Int. J. Inf. Manag..

[41]  Anil K. Jain,et al.  IARPA Janus Benchmark-B Face Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[42]  Fan Yang,et al.  Privacy-Protective-GAN for Privacy Preserving Face De-Identification , 2019, Journal of Computer Science and Technology.

[43]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[44]  Simson L. Garfinkel,et al.  De-Identification of Personal Information , 2015 .

[45]  Kanchana Thilakarathna,et al.  Security and Privacy Approaches in Mixed Reality , 2018, ACM Comput. Surv..

[46]  Athanasios V. Vasilakos,et al.  Machine learning on big data: Opportunities and challenges , 2017, Neurocomputing.

[47]  Junxing Zhang,et al.  A video encryption service based on cloud computing , 2017, 2017 7th IEEE International Conference on Electronics Information and Emergency Communication (ICEIEC).

[48]  Carlo Curino,et al.  Apache Hadoop YARN: yet another resource negotiator , 2013, SoCC.

[49]  Ramakant Nevatia,et al.  Face and Body Association for Video-Based Face Recognition , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[50]  Zhongyuan Wang,et al.  Smart Monitoring Cameras Driven Intelligent Processing to Big Surveillance Video Data , 2018, IEEE Transactions on Big Data.

[51]  Yi Han,et al.  A Fine-Grained Video Encryption Service Based on the Cloud-Fog-Local Architecture for Public and Private Videos , 2019, Sensors.

[52]  Chang Liu,et al.  PF-Face: A Parallel Framework for Face Classification and Search from Massive Videos Based on Spark , 2018, 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM).

[53]  Ju Ren,et al.  A Survey on End-Edge-Cloud Orchestrated Network Computing Paradigms , 2019, ACM Comput. Surv..

[54]  Songjie Gong,et al.  Learning User Interest Model for Content-based Filtering in Personalized Recommendation System , 2012 .

[55]  Craig Gentry,et al.  Fully homomorphic encryption using ideal lattices , 2009, STOC '09.

[56]  Richa Singh,et al.  Anonymizing k-Facial Attributes via Adversarial Perturbations , 2018, IJCAI.

[57]  William Robson Schwartz,et al.  A scalable and flexible framework for smart video surveillance , 2016, Comput. Vis. Image Underst..

[58]  M.A.P. Chamikara,et al.  Privacy Preserving Face Recognition Utilizing Differential Privacy , 2020, Comput. Secur..

[59]  Rama Chellappa,et al.  Unconstrained face verification using deep CNN features , 2015, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[60]  Ashish Ghosh,et al.  Big data analytics for video surveillance , 2019, Multimedia Tools and Applications.

[61]  Yang Li,et al.  FPGAN: Face de-identification method with generative adversarial networks for social robots , 2020, Neural Networks.

[62]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[63]  Eun-Soo Kim,et al.  Hierarchical topic modeling with pose-transition feature for action recognition using 3D skeleton data , 2018, Inf. Sci..

[64]  Richard Hill,et al.  Cloud-based scalable object detection and classification in video streams , 2018, Future Gener. Comput. Syst..

[65]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[66]  Yi Mu,et al.  Secure Fine-Grained Access Control and Data Sharing for Dynamic Groups in the Cloud , 2018, IEEE Transactions on Information Forensics and Security.

[67]  Cordelia Schmid,et al.  Dense Trajectories and Motion Boundary Descriptors for Action Recognition , 2013, International Journal of Computer Vision.

[68]  Gang Wang,et al.  NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).