PrivHAR: Recognizing Human Actions From Privacy-preserving Lens

The accelerated use of digital cameras prompts an increasing concern about privacy and security, particularly in applications such as action recognition. In this paper, we propose an optimizing framework to provide robust visual privacy protection along the human action recognition pipeline. Our framework parameterizes the camera lens to successfully degrade the quality of the videos to inhibit privacy attributes and protect against adversarial attacks while maintaining relevant features for activity recognition. We validate our approach with extensive simulations and hardware experiments.

[1]  Hailin Jin,et al.  Privacy-Preserving Deep Action Recognition: An Adversarial Learning Framework and A New Dataset , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Michael S. Bernstein,et al.  Visual Intelligence through Human Interaction , 2021, Human–Computer Interaction Series.

[4]  Juan Carlos Niebles,et al.  Learning Privacy-preserving Optics for Human Pose Estimation , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[5]  Michael S. Bernstein,et al.  On the Opportunities and Risks of Foundation Models , 2021, ArXiv.

[6]  E. Vera,et al.  Snapshot compressive spectral depth imaging from coded aberrations. , 2021, Optics express.

[7]  Ankit Thakkar,et al.  A survey on video-based Human Action Recognition: recent updates, datasets, challenges, and applications , 2020, Artif. Intell. Rev..

[8]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Jonathan Tompson,et al.  Counting Out Time: Class Agnostic Video Repetition Counting in the Wild , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Juan Carlos Niebles,et al.  Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction , 2020, IEEE Robotics and Automation Letters.

[11]  Yifan Peng,et al.  Deep Optics for Single-Shot High-Dynamic-Range Imaging , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Juan Carlos Niebles,et al.  RubiksNet: Learnable 3D-Shift for Efficient Video Action Recognition , 2020, ECCV.

[13]  Didik Purwanto,et al.  Extreme Low Resolution Action Recognition with Spatial-Temporal Multi-Head Self-Attention and Knowledge Distillation , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[14]  Kandasamy Illanko,et al.  Human Action Recognition Using Convolutional Neural Network and Depth Sensor Data , 2019, Proceedings of the 2019 International Conference on Information Technology and Computer Communications - ITCC 2019.

[15]  Zhangyang Wang,et al.  DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Henry Arguello,et al.  Compressive spectral imaging via deformable mirror and colored-mosaic detector. , 2019, Optics express.

[17]  Ahmet Gunduz,et al.  Resource Efficient 3D Convolutional Neural Networks , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[18]  Vibhav Vineet,et al.  Privacy-Preserving Action Recognition Using Coded Aperture Videos , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[19]  Bo Chen,et al.  MnasNet: Platform-Aware Neural Architecture Search for Mobile , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Ayan Chakrabarti,et al.  Learning Privacy Preserving Encodings Through Adversarial Training , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[21]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Antonis A. Argyros,et al.  Unsupervised Detection of Periodic Segments in Videos , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[23]  Stephen P. Boyd,et al.  End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging , 2018, ACM Trans. Graph..

[24]  Zhenyu Wu,et al.  Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study , 2018, ECCV.

[25]  Yong Jae Lee,et al.  Learning to Anonymize Faces for Privacy Preserving Action Detection , 2018, ECCV.

[26]  Dapeng Tao,et al.  Skeleton embedded motion body partition for human action recognition using depth sequences , 2018, Signal Process..

[27]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Michael S. Ryoo,et al.  Extreme Low Resolution Activity Recognition with Multi-Siamese Embedding Learning , 2017, AAAI.

[29]  Sanjeev J. Koppal,et al.  Pre-Capture Privacy for Small Vision Sensors , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Ivan Sikiric,et al.  I Know That Person: Generative Full Body and Face De-identification of People in Images , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[31]  Tribhuvanesh Orekondy,et al.  Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[32]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Michael S. Ryoo,et al.  Privacy-Preserving Human Activity Recognition from Extreme Low Resolution , 2016, AAAI.

[34]  Richard P. Wildes,et al.  Spatiotemporal Residual Networks for Video Action Recognition , 2016, NIPS.

[35]  Luc Van Gool,et al.  Temporal Segment Networks: Towards Good Practices for Deep Action Recognition , 2016, ECCV.

[36]  Yuxiao Hu,et al.  MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition , 2016, ECCV.

[37]  Nikos Komodakis,et al.  Wide Residual Networks , 2016, BMVC.

[38]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  M. Schmid Principles Of Optics Electromagnetic Theory Of Propagation Interference And Diffraction Of Light , 2016 .

[41]  Lynne M Connelly,et al.  Fisher's Exact Test. , 2016, Medsurg nursing : official journal of the Academy of Medical-Surgical Nurses.

[42]  Sanjeev J. Koppal,et al.  Privacy preserving optics for miniature vision sensors , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Alexandros André Chaaraoui,et al.  Visual privacy protection methods: A survey , 2015, Expert Syst. Appl..

[44]  Hassan Foroosh,et al.  Exploring Sparseness and Self-Similarity for Action Recognition , 2015, IEEE Transactions on Image Processing.

[45]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[46]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Nasser Kehtarnavaz,et al.  Real-time human action recognition based on depth motion maps , 2013, Journal of Real-Time Image Processing.

[48]  Honglak Lee,et al.  Learning to Align from Scratch , 2012, NIPS.

[49]  Thomas Serre,et al.  HMDB: A large video database for human motion recognition , 2011, 2011 International Conference on Computer Vision.

[50]  Vasudevan Lakshminarayanan,et al.  Zernike polynomials: a guide , 2011 .

[51]  P. J. Narayanan,et al.  Person De-Identification in Videos , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[52]  Patrick Pérez,et al.  View-Independent Action Recognition from Temporal Self-Similarities , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Djemel Ziou,et al.  Image Quality Metrics: PSNR vs. SSIM , 2010, 2010 20th International Conference on Pattern Recognition.

[54]  Rong Yan,et al.  Tools for Protecting the Privacy of Specific Individuals in Video , 2007, EURASIP J. Adv. Signal Process..

[55]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[56]  J. Goodman Introduction to Fourier optics , 1969 .