Classification of Soft Keyboard Typing Behaviors Using Mobile Device Sensors with Machine Learning

The amount of personal data stored on mobile devices has risen significantly during the past several years as a result of two developments: More people are using them, and sensors have become more advanced, capable of analyzing and classifying human activities such as walking, running, sleeping and cycling, and swimming. In this study, we propose a system to classify users’ typing behaviors based on the data produced by the built-in sensors and present a login use case scenario to validate the results. We investigate users’ unique typing and phone holding behaviors by examining the soft biometric (age, gender) and statistical features. Typing behaviors are classified by various machine learning techniques with the data inputted from accelerometer and gyroscope sensors. Artificial neural networks (ANN), k-nearest neighbors (k-NN), support vector machines (SVM) and RandomForest Classifier (RFC) algorithms, which are some of the most common algorithms, were applied for classification. In the user studies, we achieved accuracy of 98.55% for ANN, 100% for k-NN, 99.8% for SVM and 99.5% for RFC. The system is capable of device-based training and can distinguish the device owner’s typing behavior from those of others with 100% accuracy. The proposed system was tested on a developed mobile application prototype, and its applicability was shown through experiments.

[1]  Gustavo A. Ramírez González,et al.  A mobile system for sedentary behaviors classification based on accelerometer and location data , 2017, Comput. Ind..

[2]  David A. Wagner,et al.  Are You Ready to Lock? , 2014, CCS.

[3]  Miroslav Kubat Artificial Neural Networks , 2015 .

[4]  Dinh Q. Phung,et al.  Nonparametric discovery of movement patterns from accelerometer signals , 2016, Pattern Recognit. Lett..

[5]  Bernt Schiele,et al.  ADL recognition based on the combination of RFID and accelerometer sensing , 2008, Pervasive 2008.

[6]  Imran A. Zualkernan,et al.  Emotion recognition using mobile phones , 2016, 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom).

[7]  L. A. Smith,et al.  Feature Subset Selection: A Correlation Based Filter Approach , 1997, ICONIP.

[8]  Scott Campbell,et al.  Using Accelerometer and Gyroscopic Measures to Quantify Postural Stability. , 2015, Journal of athletic training.

[9]  Bogdan Kwolek,et al.  Improving fall detection by the use of depth sensor and accelerometer , 2015, Neurocomputing.

[10]  Hugo Gamboa,et al.  Human activity data discovery from triaxial accelerometer sensor: Non-supervised learning sensitivity to feature extraction parametrization , 2015, Inf. Process. Manag..

[11]  Dongsong Zhang,et al.  Harmonized authentication based on ThumbStroke dynamics on touch screen mobile phones , 2016, Decis. Support Syst..

[12]  Johannes Willem Joubert,et al.  Combining accelerometer data and contextual variables to evaluate the risk of driver behaviour , 2016 .

[13]  Lars Kotthoff,et al.  Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA , 2017, J. Mach. Learn. Res..

[14]  Francesc Soriguera,et al.  DRIVER FEEDBACK MOBILE APP , 2016 .

[15]  Yuji Watanabe,et al.  Toward an Immunity-based Gait Recognition on Smart Phone: A Study of Feature Selection and Walking State Classification , 2016, KES.

[16]  Cagatay Catal,et al.  On the use of ensemble of classifiers for accelerometer-based activity recognition , 2015, Appl. Soft Comput..

[17]  Tatsuo Nakajima,et al.  Feature Selection and Activity Recognition from Wearable Sensors , 2006, UCS.

[18]  Corina Nüesch,et al.  Mobile inertial sensor based gait analysis: Validity and reliability of spatiotemporal gait characteristics in healthy seniors. , 2016, Gait & posture.

[19]  Lior Rokach,et al.  Anomaly detection for smartphone data streams , 2017, Pervasive Mob. Comput..

[20]  Meng-Shiuan Pan,et al.  Using accelerometer for counting and identifying swimming strokes , 2016, Pervasive Mob. Comput..

[21]  Feng Hao,et al.  TouchSignatures: Identification of user touch actions and PINs based on mobile sensor data via JavaScript , 2016, J. Inf. Secur. Appl..

[22]  Muhammad Zubair Asghar,et al.  COGEMO: Cognitive-Based Emotion Detection from Patient Generated Health Reviews , 2017 .

[23]  Daniel Vogel,et al.  Evaluating authentication options for mobile health applications in younger and older adults , 2018, PloS one.

[24]  Andreu Català,et al.  Posture transition identification on PD patients through a SVM-based technique and a single waist-worn accelerometer , 2015, Neurocomputing.

[25]  Olcay Sahin,et al.  Modeling the impact of latent driving patterns on traffic safety using mobile sensor data. , 2017, Accident; analysis and prevention.

[26]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[27]  Mahesh Pal,et al.  Random forest classifier for remote sensing classification , 2005 .

[28]  Luis González Abril,et al.  Mobile activity recognition and fall detection system for elderly people using Ameva algorithm , 2017, Pervasive Mob. Comput..

[29]  Jie Liu,et al.  Sensor-Based User Authentication , 2015, EWSN.

[30]  Donald Y. C. Lie,et al.  A real-time fall detection system using a wearable gait analysis sensor and a Support Vector Machine (SVM) classifier , 2015, 2015 Eighth International Conference on Mobile Computing and Ubiquitous Networking (ICMU).

[31]  Matti Pietikäinen,et al.  Face and Eye Detection for Person Authentication in Mobile Phones , 2007, 2007 First ACM/IEEE International Conference on Distributed Smart Cameras.

[32]  Heinrich Hußmann,et al.  I Feel Like I'm Taking Selfies All Day!: Towards Understanding Biometric Authentication on Smartphones , 2015, CHI.

[33]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[34]  Alexander J. Casson,et al.  Gyroscope vs. accelerometer measurements of motion from wrist PPG during physical exercise , 2016, ICT Express.

[35]  Molly K. Maskrey,et al.  Using Machine Learning , 2022, Pro iPhone Development with SwiftUI.

[36]  Rana Fayyaz Ahmad,et al.  Feature extraction and classification for EEG signals using wavelet transform and machine learning techniques , 2015, Australasian Physical & Engineering Sciences in Medicine.

[37]  Anis Koubâa,et al.  RoadSense: Smartphone Application to Estimate Road Conditions Using Accelerometer and Gyroscope , 2017, IEEE Sensors Journal.

[38]  Rama Chellappa,et al.  Facial attributes for active authentication on mobile devices , 2017, Image Vis. Comput..

[39]  Blase Ur,et al.  Biometric authentication on iPhone and Android: Usability, perceptions, and influences on adoption , 2015 .

[40]  P. Muhammad,et al.  Hand Gesture User Interface for Smart Devices Based on Mems Sensors , 2016 .

[41]  Andrea Mannini,et al.  Activity recognition using a single accelerometer placed at the wrist or ankle. , 2013, Medicine and science in sports and exercise.

[42]  Mikael Forsman,et al.  An iPhone application for upper arm posture and movement measurements. , 2017, Applied ergonomics.

[43]  Xiaohong Guan,et al.  Input extraction via motion-sensor behavior analysis on smartphones , 2015, Comput. Secur..

[44]  Abbes Amira,et al.  Multiple comparator classifier framework for accelerometer-based fall detection and diagnostic , 2016, Appl. Soft Comput..

[45]  Kim-Phuong L. Vu,et al.  The Influence of Password Restrictions and Mnemonics on the Memory for Passwords of Older Adults , 2013, HCI.

[46]  Orfeu M Buxton,et al.  Practical considerations in using accelerometers to assess physical activity, sedentary behavior, and sleep. , 2015, Sleep health.

[47]  Muhammad Zubair Asghar,et al.  A Rule-Based Sentiment Classification Framework for Health Reviews on Mobile Social Media , 2017 .

[48]  Seema Verma,et al.  A survey on driver behavior detection techniques for intelligent transportation systems , 2017, 2017 7th International Conference on Cloud Computing, Data Science & Engineering - Confluence.

[49]  Zoran A. Salcic,et al.  Adaptive sliding window segmentation for physical activity recognition using a single tri-axial accelerometer , 2017, Pervasive Mob. Comput..

[50]  Ning Zhang,et al.  A survey on touch dynamics authentication in mobile devices , 2016, Comput. Secur..

[51]  Christoforos Ntantogian,et al.  Gaithashing: A two-factor authentication scheme based on gait features , 2015, Comput. Secur..

[52]  Abeer Alsadoon,et al.  Designing Touch-Based Hybrid Authentication Method for Smartphones , 2015 .

[53]  H. Kishino,et al.  Potential of a smartphone as a stress-free sensor of daily human behaviour , 2015, Behavioural Brain Research.