Privacy-preserving Pedestrian Tracking using Distributed 3D LiDARs

The growing demand for intelligent environments unleashes an extraordinary cycle of privacy-aware applications that makes individuals' life more comfortable and safe. Examples of these applications include pedestrian tracking systems in large areas. Although the ubiquity of camera-based systems, they are not a preferable solution due to the vulnerability of leaking the privacy of pedestrians. In this paper, we introduce a novel privacy-preserving system for pedestrian tracking in smart environments using multiple distributed LiDARs of non-overlapping views. The system is designed to leverage LiDAR devices to track pedestrians in partially covered areas due to practical constraints, e.g., occlusion or cost. Therefore, the system uses the point cloud captured by different LiDARs to extract discriminative features that are used to train a metric learning model for pedestrian matching purposes. To boost the system's robustness, we leverage a probabilistic approach to model and adapt the dynamic mobility patterns of individuals and thus connect their sub-trajectories. We deployed the system in a large-scale testbed with 70 colorless LiDARs and conducted three different experiments. The evaluation result at the entrance hall confirms the system's ability to accurately track the pedestrians with a 0.98 F-measure even with zero-covered areas. This result highlights the promise of the proposed system as the next generation of privacy-preserving tracking means in smart environments.

[1]  H. Yamaguchi,et al.  Demonstrating hitonavi-μ: a novel wearable LiDAR for human activity recognition , 2022, MobiCom.

[2]  H. Yamaguchi,et al.  Object Recognition from 3D Point Cloud on Resource-Constrained Edge Device , 2022, 2022 18th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob).

[3]  H. Yamaguchi,et al.  On-the-Fly Spatio-Temporal Human Segmentation of 3D Point Cloud Data By Micro-Size LiDAR , 2022, 2022 18th International Conference on Intelligent Environments (IE).

[4]  Maged A. Youssef,et al.  Laser Range Scanners for Enabling Zero-overhead WiFi-based Indoor Localization System , 2022, ACM Trans. Spatial Algorithms Syst..

[5]  H. Yamaguchi,et al.  An Accurate Point Cloud-Based Human Identification Using Micro-Size LiDAR , 2022, 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops).

[6]  H. Yamaguchi,et al.  You Work We Care: Sitting Posture Assessment Based on Point Cloud Data , 2022, 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops).

[7]  Akihito Hiromori,et al.  Pedestrian Tracking in Public Passageway by Single 3D Depth Sensor , 2022, 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops).

[8]  Meibin Qi,et al.  An End-to-end Heterogeneous Restraint Network for RGB-D Cross-modal Person Re-identification , 2022, ACM Trans. Multim. Comput. Commun. Appl..

[9]  Juan Carlos Niebles,et al.  Learning Privacy-preserving Optics for Human Pose Estimation , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Jonathan P. How,et al.  Human Trajectory Prediction Using Similarity-Based Multi-Model Fusion , 2021, IEEE Robotics and Automation Letters.

[11]  Tianxu Xu,et al.  A Review: Point Cloud-Based 3D Human Joints Estimation , 2021, Sensors.

[12]  Cheng Wang,et al.  3D Multi-Object Tracking in Point Clouds Based on Prediction Confidence-Guided Data Association , 2021, IEEE Transactions on Intelligent Transportation Systems.

[13]  Hirozumi Yamaguchi,et al.  Gain Without Pain: Enabling Fingerprinting-based Indoor Localization using Tracking Scanners , 2020, SIGSPATIAL/GIS.

[14]  Stefan Roth,et al.  MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking , 2020, International Journal of Computer Vision.

[15]  Hongsheng Li,et al.  Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive Object Re-ID , 2020, NeurIPS.

[16]  Daniel Cremers,et al.  MOT20: A benchmark for multi object tracking in crowded scenes , 2020, ArXiv.

[17]  Silvio Savarese,et al.  JRMOT: A Real-Time 3D Multi-Object Tracker and a New Large-Scale Dataset , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Yang Yang,et al.  RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  Vicente Matellán Olivera,et al.  People Detection and Tracking Using LIDAR Sensors , 2019, Robotics.

[20]  David Held,et al.  3D Multi-Object Tracking: A Baseline and New Evaluation Metrics , 2019, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Horst-Michael Groß,et al.  Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Jenq-Neng Hwang,et al.  CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Trevor Darrell,et al.  Joint Monocular 3D Vehicle Detection and Tracking , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[24]  Raquel Urtasun,et al.  End-to-end Learning of Multi-sensor 3D Tracking by Detection , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Carlo Tomasi,et al.  Features for Multi-target Multi-camera Tracking and Re-identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Mohan M. Trivedi,et al.  No Blind Spots: Full-Surround Multi-Object Tracking for Autonomous Vehicles Using Cameras and LiDARs , 2018, IEEE Transactions on Intelligent Vehicles.

[27]  Anath Fischer,et al.  3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks , 2017, ArXiv.

[28]  Sanjeev J. Koppal,et al.  Pre-Capture Privacy for Small Vision Sensors , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Jian-Huang Lai,et al.  Robust Depth-Based Person Re-Identification , 2017, IEEE Transactions on Image Processing.

[30]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[31]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Vitaly Shmatikov,et al.  Membership Inference Attacks Against Machine Learning Models , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[33]  Qi Tian,et al.  MARS: A Video Benchmark for Large-Scale Person Re-Identification , 2016, ECCV.

[34]  Michel Bierlaire,et al.  Probabilistic speed–density relationship for pedestrian traffic , 2016 .

[35]  Jonathan P. How,et al.  Augmented dictionary learning for motion prediction , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Qi Tian,et al.  Scalable Person Re-identification: A Benchmark , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[37]  Somesh Jha,et al.  Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures , 2015, CCS.

[38]  Zhiyu Xiang,et al.  Object tracking with 3D LIDAR via multi-task sparse learning , 2015, 2015 IEEE International Conference on Mechatronics and Automation (ICMA).

[39]  Shaojie Qiao,et al.  TraPlan: An Effective Three-in-One Trajectory-Prediction Model in Transportation Networks , 2015, IEEE Transactions on Intelligent Transportation Systems.

[40]  Yang Song,et al.  Learning Fine-Grained Image Similarity with Deep Ranking , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Markus Maurer,et al.  Multi-Target Tracking using a 3D-Lidar sensor for autonomous vehicles , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[42]  Sergio Escalera,et al.  Tri-modal Person Re-identification with RGB, Depth and Thermal Features , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[43]  F. Perronnin,et al.  Image Classification with the Fisher Vector: Theory and Practice , 2013, International Journal of Computer Vision.

[44]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[46]  Shin'ichi Yuta,et al.  People detection using range and intensity data from multi-layered Laser Range Finders , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[47]  Joel A. Hesch,et al.  Tracking People with a 360-Degree Lidar , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[48]  Roland Siegwart,et al.  A Layered Approach to People Detection in 3D Range Data , 2010, AAAI.

[49]  François Bourgeois,et al.  An extension of the Munkres algorithm for the assignment problem to rectangular matrices , 1971, CACM.

[50]  J. Munkres ALGORITHMS FOR THE ASSIGNMENT AND TRANSIORTATION tROBLEMS* , 1957 .

[51]  Jenq-Neng Hwang,et al.  Multi-Camera Tracking of Vehicles based on Deep Features Re-ID and Trajectory-Based Camera Link Models , 2019, CVPR Workshops.