ViPED: On-road vehicle passenger detection for autonomous vehicles

Abstract This paper is about detecting and counting the passengers of a tracking vehicle using on-car monocular vision. By having a model of nearby vehicle occupants, intelligent reasoning systems of autonomous cars will be provided with this additional knowledge needed in emergency situations such as those that many philosophers have recently raised. The on-road Vehicle PassengEr Detection (ViPED) system is based on the human perception model in terms of spatio-temporal reasoning, namely the slight movements of passenger shape silhouettes inside the cabin. The main challenges we face are the low light conditions of the cabin (no feature points), the subtle non-rigid motions of the occupants (possible artifactual transitions), and the puzzling discrimination problem of back or front seat occupants (lack of depth information inside the cabin). To overcome these challenges, we first track the detected car windshield and find the optimal affine warp. The registered windshield images are preprocessed in order to extract a feature matrix, which serves as input to a Convolutional Neural Network (CNN) for inferring the number and position of passengers. We demonstrate that our low-cost sensor system is able to detect in most cases successfully all the passengers in preceding moving vehicles at various distances and occupancies. Metrics and datasets are included for possible community future work on this new challenging task.

[1]  Vassilios Morellas,et al.  A vehicle occupant counting system based on near-infrared phenomenology and fuzzy neural classification , 2000, IEEE Trans. Intell. Transp. Syst..

[2]  Nikolaos Papanikolopoulos,et al.  Automatic detection of vehicle occupants: the imaging problemand its solution , 2000, Machine Vision and Applications.

[3]  Zhiwei Song,et al.  Appearance-based Brake-Lights recognition using deep learning and vehicle detection , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  Tarak Gandhi,et al.  Looking-In and Looking-Out of a Vehicle: Computer-Vision-Based Enhanced Vehicle Safety , 2007, IEEE Transactions on Intelligent Transportation Systems.

[6]  Mubarak Shah,et al.  Landing a UAV on a runway using image registration , 2008, 2008 IEEE International Conference on Robotics and Automation.

[7]  Philip Birch,et al.  Automated vehicle occupancy monitoring , 2004 .

[8]  Xiaoou Tang,et al.  A large-scale car dataset for fine-grained categorization and verification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Huadong Ma,et al.  Robust Head-Shoulder Detection by PCA-Based Multilevel HOG-LBP Detector for People Counting , 2010, 2010 20th International Conference on Pattern Recognition.

[10]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Orhan Bulan,et al.  Passenger Compartment Violation Detection in HOV/HOT Lanes , 2016, IEEE Transactions on Intelligent Transportation Systems.

[12]  Huiji Gao,et al.  Harnessing the Crowdsourcing Power of Social Media for Disaster Relief , 2011, IEEE Intelligent Systems.

[13]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Martin Lauer,et al.  3D Traffic Scene Understanding From Movable Platforms , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Robert P. Loce,et al.  A machine learning approach for detecting cell phone usage , 2015, Electronic Imaging.

[16]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[18]  Xiaolei Ma,et al.  Self-Adaptive Tolling Strategy for Enhanced High-Occupancy Toll Lane Operations , 2014, IEEE Transactions on Intelligent Transportation Systems.

[19]  Luis Miguel Bergasa,et al.  Occupant Monitoring System for Traffic Control Based on Visual Categorization , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[20]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[21]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[22]  Roman P. Pflugfelder,et al.  Clustering of static-adaptive correspondences for deformable object tracking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Xue,et al.  A vehicle occupant counting system using near-infrared (NIR) image , 2012, 2012 IEEE 11th International Conference on Signal Processing.

[25]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[26]  Daniel P. Huttenlocher,et al.  Pictorial Structures for Object Recognition , 2004, International Journal of Computer Vision.

[27]  Mario Gerla,et al.  Internet of vehicles: From intelligent grid to autonomous cars and vehicular clouds , 2014, 2014 IEEE World Forum on Internet of Things (WF-IoT).

[28]  P. Foot.  The Problem of Abortion and the Doctrine of the Double Effect , 2020, The Doctrine of Double Effect.

[29]  Florent Perronnin,et al.  A machine learning approach to vehicle occupancy detection , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[30]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[31]  Paul A. Viola,et al.  Fast and Robust Classification using Asymmetric AdaBoost and a Detector Cascade , 2001, NIPS.