Manifold learning for user profiling and identity verification using motion sensors

Abstract Mobile devices are becoming ubiquitous and being increasingly used for data-sensitive activities such as communication, personal media storage, and banking. The protection of such data commonly relies on passwords and biometric traits such as fingerprints. These methods perform the user authentication sporadically and often require action from the user, which may make them susceptible to spoofing attacks. This scenario can be mitigated if we bring to bear motion-sensing based methods for authentication, which operate continuously and without requiring user action, hence are harder to attack. Such methods could be used allied with traditional authentication methods or on their own. This paper explores this idea in a novel user-agnostic approach for identity verification based on motion traits acquired by mobile sensors. The proposed approach does not require user-specific training before deployment in mobile devices nor does it require any extra sensor in the device. This solution is capable of learning a user profiling manifold from a small user subset and extend it to unknown users. We validated the proposal on two public datasets. The reported experiments demonstrate remarkable results under a cross-dataset protocol and an open-set setup. Moreover, we performed several analyses aiming at answering critical questions of a biometric method and the presented solution.

[1]  Chunheng Wang,et al.  Deep nonlinear metric learning with independent subspace analysis for face verification , 2012, ACM Multimedia.

[2]  Rama Chellappa,et al.  Visual Domain Adaptation: A survey of recent advances , 2015, IEEE Signal Processing Magazine.

[3]  Jiwen Lu,et al.  Uncorrelated discriminant simplex analysis for view-invariant gait signal computing , 2010, Pattern Recognit. Lett..

[4]  Joseph Hamill,et al.  Biomechanical Basis of Human Movement , 1995 .

[5]  Thuc Dinh Nguyen,et al.  Adaptive Cross-Device Gait Recognition Using a Mobile Accelerometer , 2013, J. Inf. Process. Syst..

[6]  Yasushi Makihara,et al.  The largest inertial sensor-based gait database and performance evaluation of gait-based personal authentication , 2014, Pattern Recognit..

[7]  Arun Ross,et al.  50 years of biometric research: Accomplishments, challenges, and opportunities , 2016, Pattern Recognit. Lett..

[8]  Anderson Rocha,et al.  Toward Open Set Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[10]  Maria De Marsico,et al.  Embedded Accelerometer Signal Normalization for Cross-Device Gait Recognition , 2016, 2016 International Conference of the Biometrics Special Interest Group (BIOSIG).

[11]  Mark J. Shensa,et al.  The discrete wavelet transform: wedding the a trous and Mallat algorithms , 1992, IEEE Trans. Signal Process..

[12]  Ingo Stengel,et al.  Impact of External Parameters on the Gait Recognition Using a Smartphone , 2015, ICISSP.

[13]  Suiping Zhou,et al.  Wearable Device-Based Gait Recognition Using Angle Embedded Gait Dynamic Images and a Convolutional Neural Network , 2017, Sensors.

[14]  Christoph Busch,et al.  Classification of Acceleration Data for Biometric Gait Recognition on Mobile Devices , 2011, BIOSIG.

[15]  Paul J. M. Havinga,et al.  A Survey of Online Activity Recognition Using Mobile Phones , 2015, Sensors.

[16]  Gustau Camps-Valls,et al.  Kernel Manifold Alignment for Domain Adaptation , 2015, PloS one.

[17]  Siome Goldenstein,et al.  User-Centric Coordinates for Applications Leveraging 3-Axis Accelerometer Data , 2017, IEEE Sensors Journal.

[18]  Christoph Busch,et al.  Authentication of Smartphone Users Based on the Way They Walk Using k-NN Algorithm , 2012, 2012 Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[19]  Debi Prosad Dogra,et al.  Multimodal Gait Recognition With Inertial Sensor Data and Video Using Evolutionary Algorithm , 2019, IEEE Transactions on Fuzzy Systems.

[20]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[21]  Thuc Dinh Nguyen,et al.  A Generalized Authentication Scheme for Mobile Phones Using Gait Signals , 2015, ICETE.

[22]  D. Cunningham,et al.  Age-related changes in speed of walking. , 1988 .

[23]  Xuelong Li,et al.  Discriminant Locally Linear Embedding With High-Order Tensor Data , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Vir V. Phoha,et al.  A Survey on Gait Recognition , 2018, ACM Comput. Surv..

[25]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[26]  Dong Li,et al.  A method of anomaly detection and fault diagnosis with online adaptive learning under small training samples , 2017, Pattern Recognit..

[27]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Ricardo da Silva Torres,et al.  Semi-supervised transfer subspace for domain adaptation , 2018, Pattern Recognit..

[29]  Chang-Tsun Li,et al.  Accelerometer Dense Trajectories for Activity Recognition and People Identification , 2019, 2019 7th International Workshop on Biometrics and Forensics (IWBF).

[30]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[31]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[32]  Rubén San-Segundo-Hernández,et al.  Frequency features and GMM-UBM approach for gait-based person identification using smartphone inertial signals , 2016, Pattern Recognit. Lett..

[33]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[34]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[35]  Matteo Gadaleta,et al.  IDNet: Smartphone-based Gait Recognition with Convolutional Neural Networks , 2016, Pattern Recognit..

[36]  Arun Ross,et al.  Biometric recognition by gait: A survey of modalities and features , 2018, Comput. Vis. Image Underst..

[37]  Elin Kolle,et al.  Accelerometer-determined physical activity and self-reported health in a population of older adults (65–85 years): a cross-sectional study , 2014, BMC Public Health.

[38]  Heikki Ailisto,et al.  Identifying users of portable devices from gait pattern with accelerometers , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[39]  Mikko Lindholm,et al.  Identifying people from gait pattern with accelerometers , 2005, SPIE Defense + Commercial Sensing.

[40]  Fabio Martinelli,et al.  Try Walking in My Shoes, if You Can: Accurate Gait Recognition Through Deep Learning , 2017, SAFECOMP Workshops.

[41]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[42]  Devu Manikantan Shila,et al.  Adversarial Gait Detection on Mobile Devices Using Recurrent Neural Networks , 2018, 2018 17th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/ 12th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE).

[43]  Gary M. Weiss,et al.  Cell phone-based biometric identification , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[44]  G. Johansson Visual perception of biological motion and a model for its analysis , 1973 .

[45]  Christoph Busch,et al.  Benchmarking the performance of SVMs and HMMs for accelerometer-based biometric gait recognition , 2011, 2011 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[46]  Khaled H. Hamed,et al.  Time-frequency analysis , 2003 .

[47]  S. Sprager,et al.  A cumulant-based method for gait identification using accelerometer data with principal component analysis and support vector machine , 2009 .

[48]  Mohammad Esmalifalak,et al.  A data mining approach for fault diagnosis: An application of anomaly detection algorithm , 2014 .

[49]  Rudolf Fleischer,et al.  Distance Approximating Dimension Reduction of Riemannian Manifolds , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[50]  Tianjian Ji,et al.  FREQUENCY AND VELOCITY OF PEOPLE WALKING , 2005 .