Identity Recognition in Intelligent Cars with Behavioral Data and LSTM-ResNet Classifier

Identity recognition in a car cabin is a critical task nowadays and offers a great field of applications ranging from personalizing intelligent cars to suit drivers physical and behavioral needs to increasing safety and security. However, the performance and applicability of published approaches are still not suitable for use in series cars and need to be improved. In this paper, we investigate Human Identity Recognition in a car cabin with Time Series Classification (TSC) and deep neural networks. We use gas and brake pedal pressure as input to our models. This data is easily collectable during driving in everyday situations. Since our classifiers have very little memory requirements and do not require any input data preproccesing, we were able to train on one Intel i5-3210M processor only. Our classification approach is based on a combination of LSTM and ResNet. The network trained on a subset of NUDrive outperforms the ResNet and LSTM models trained solely by 35.9 % and 53.85 % accuracy respectively. We reach a final accuracy of 79.49 % on a 10-drivers subset of NUDrive and 96.90 % on a 5-drivers subset of UTDrive.

[1]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Xiang Li,et al.  Understanding the Disharmony Between Dropout and Batch Normalization by Variance Shift , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Abdenour Hadid,et al.  Face Biometrics Under Spoofing Attacks: Vulnerabilities, Countermeasures, Open Issues, and Research Directions , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[6]  Javier Echanobe,et al.  Driving Behavior Signals and Machine Learning: A Personalized Driver Assistance System , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[7]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Jean-Luc Dugelay,et al.  On the vulnerability of face recognition systems to spoofing mask attacks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[9]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[10]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[11]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[13]  Bernd Ludwig,et al.  InCarMusic: Context-Aware Music Recommendations in a Car , 2011, EC-Web.

[14]  Driss Aboutajdine,et al.  Boosting 3-D-Geometric Features for Efficient Face Recognition and Gender Classification , 2012, IEEE Transactions on Information Forensics and Security.

[15]  Matthew J. Hausknecht,et al.  Beyond short snippets: Deep networks for video classification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Zahid Akhtar,et al.  Face Recognition Systems under Spoofing Attack , 2011 .

[17]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[18]  Hakan Erdogan,et al.  Experiments on decision fusion for driver recognition , 2007 .

[19]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Hakan Erdogan,et al.  Multi-modal Person Recognition for Vehicular Applications , 2005, Multiple Classifier Systems.

[21]  Patrick Schäfer.  The BOSS is concerned with time series classification in the presence of noise , 2014, Data Mining and Knowledge Discovery.

[22]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[23]  Mark B. Sandler,et al.  Text-based LSTM networks for Automatic Music Composition , 2016, ArXiv.

[24]  Gerald Penn,et al.  Convolutional Neural Networks for Speech Recognition , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[25]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Tim Oates,et al.  Time series classification from scratch with deep neural networks: A strong baseline , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[27]  John H. L. Hansen,et al.  UTDrive: The Smart Vehicle Project , 2009 .

[28]  M. Taner Eskil,et al.  Driver Recognition Using Gaussian Mixture Models and Decision Fusion Techniques , 2008, ISICA.

[29]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[30]  Shihong Lao,et al.  3D template matching for pose invariant face recognition using 3D facial model built with isoluminance line based stereo vision , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[31]  John H. L. Hansen,et al.  In-Vehicle Corpus and Signal Processing for Driver Behavior , 2008 .

[32]  Sébastien Marcel,et al.  Face Recognition Systems Under Spoofing Attacks , 2016, Face Recognition Across the Imaging Spectrum.

[33]  Xiaogang Wang,et al.  Hybrid Deep Learning for Face Verification , 2013, 2013 IEEE International Conference on Computer Vision.

[34]  Germain Forestier,et al.  Deep learning for time series classification: a review , 2018, Data Mining and Knowledge Discovery.

[35]  Mark Poguntke,et al.  The Personal Adaptive In-Car HMI: Integration of External Applications for Personalized Use , 2011, UMAP Workshops.

[36]  Linda Ng Boyle,et al.  Taxonomy of Mitigation Strategies for Driver Distraction , 2003 .

[37]  Xiaogang Wang,et al.  DeepID3: Face Recognition with Very Deep Neural Networks , 2015, ArXiv.

[38]  K. Itou,et al.  Driver Identification Based on Spectral Analysis of Driving Behavioral Signals , 2007 .

[39]  Kazuya Takeda,et al.  CIAIR In-Car Speech Corpus - Influence of Driving Status , 2005, IEICE Trans. Inf. Syst..

[40]  Carlos Busso,et al.  Driver Modeling for Detection and Assessment of Driver Distraction: Examples from the UTDrive Test Bed , 2017, IEEE Signal Processing Magazine.

[41]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[42]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[43]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  George C. Runger,et al.  A Bag-of-Features Framework to Classify Time Series , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  John H. L. Hansen,et al.  International Large-Scale Vehicle Corpora for Research on Driver Behavior on the Road , 2011, IEEE Transactions on Intelligent Transportation Systems.