Walking Recognition in Mobile Devices

Presently, smartphones are used more and more for purposes that have nothing to do with phone calls or simple data transfers. One example is the recognition of human activity, which is relevant information for many applications in the domains of medical diagnosis, elderly assistance, indoor localization, and navigation. The information captured by the inertial sensors of the phone (accelerometer, gyroscope, and magnetometer) can be analyzed to determine the activity performed by the person who is carrying the device, in particular in the activity of walking. Nevertheless, the development of a standalone application able to detect the walking activity starting only from the data provided by these inertial sensors is a complex task. This complexity lies in the hardware disparity, noise on data, and mostly the many movements that the smartphone can experience and which have nothing to do with the physical displacement of the owner. In this work, we explore and compare several approaches for identifying the walking activity. We categorize them into two main groups: the first one uses features extracted from the inertial data, whereas the second one analyzes the characteristic shape of the time series made up of the sensors readings. Due to the lack of public datasets of inertial data from smartphones for the recognition of human activity under no constraints, we collected data from 77 different people who were not connected to this research. Using this dataset, which we published online, we performed an extensive experimental validation and comparison of our proposals.

[1]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[2]  Lama Nachman,et al.  Unobtrusive gait verification for mobile phones , 2014, SEMWEB.

[3]  Robert Tibshirani,et al.  Estimating the number of clusters in a data set via the gap statistic , 2000 .

[4]  Jing Liu,et al.  Survey of Wireless Indoor Positioning Techniques and Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[5]  Robert Harle,et al.  A Survey of Indoor Inertial Positioning Systems for Pedestrians , 2013, IEEE Communications Surveys & Tutorials.

[6]  Dimitrios Gunopulos,et al.  Discovering similar multidimensional trajectories , 2002, Proceedings 18th International Conference on Data Engineering.

[7]  Thomas Philip Runarsson,et al.  Support vector machines and dynamic time warping for time series , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[8]  Xose Manuel Pardo,et al.  Automatic Selection of User Samples for a Non-collaborative Face Verification System , 2017, ROBOT.

[9]  Aixia Guo,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2014 .

[10]  Gaurav S. Sukhatme,et al.  Toward free-living walking speed estimation using Gaussian Process-based Regression with on-body accelerometers and gyroscopes , 2010, 2010 4th International Conference on Pervasive Computing Technologies for Healthcare.

[11]  L. Hubert,et al.  Measuring the Power of Hierarchical Cluster Analysis , 1975 .

[12]  Daniele Borio,et al.  Accelerometer Signal Features and Classification Algorithms for Positioning Applications , 2011 .

[13]  Mark A. Hall,et al.  Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning , 1999, ICML.

[14]  Feng Zhao,et al.  A reliable and accurate indoor localization method using phone inertial sensors , 2012, UbiComp.

[15]  Klaus Obermayer,et al.  Classification on Pairwise Proximity Data , 1998, NIPS.

[16]  Claus Bahlmann,et al.  Online handwriting recognition with support vector machines - a kernel approach , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[17]  Sebastian Tilch,et al.  Survey of optical indoor positioning systems , 2011, 2011 International Conference on Indoor Positioning and Indoor Navigation.

[18]  Peter Rossmanith,et al.  Simulated Annealing , 2008, Taschenbuch der Algorithmen.

[19]  T. Caliński,et al.  A dendrite method for cluster analysis , 1974 .

[20]  Roberto Iglesias,et al.  Inertial Navigation with Mobile Devices: A Robust Step Count Model , 2017, ROBOT.

[21]  Agata Brajdic,et al.  Walk detection and step counting on unconstrained smartphones , 2013, UbiComp.

[22]  H. Kriegel,et al.  Activity Recognition on 3 D Accelerometer Data ( Technical Report ) , 2013 .

[23]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[24]  Henk L. Muller,et al.  Low Cost Indoor Positioning System , 2001, UbiComp.

[25]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[26]  Toni Giorgino,et al.  Computing and Visualizing Dynamic Time Warping Alignments in R: The dtw Package , 2009 .

[27]  Gearóid ÓLaighin,et al.  When a Step Is Not a Step! Specificity Analysis of Five Physical Activity Monitors , 2017, PloS one.

[28]  L. Hubert,et al.  A general statistical framework for assessing categorical clustering in free recall. , 1976 .

[29]  Evon M. O. Abu-Taieh,et al.  Comparative Study , 2020, Definitions.

[30]  Christopher M. Bishop,et al.  Neural Network for Pattern Recognition , 1995 .

[31]  R. Tibshirani,et al.  Combining Estimates in Regression and Classification , 1996 .

[32]  A. Hof,et al.  Displacement of the pelvis during human walking: experimental data and model predictions , 1997 .

[33]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[34]  Kenneth Meijer,et al.  Activity identification using body-mounted sensors—a review of classification techniques , 2009, Physiological measurement.

[35]  Joseph Jankovic,et al.  Clinical gait and balance scale (GABS): validation and utilization , 2004, Journal of the Neurological Sciences.

[36]  J. Kuipers Quaternions and Rotation Sequences , 1998 .

[37]  Eamonn J. Keogh,et al.  Derivative Dynamic Time Warping , 2001, SDM.

[38]  Jun Yang,et al.  Physical Activity Recognition with Mobile Phones: Challenges, Methods, and Applications , 2010 .

[39]  Saibal Dutta,et al.  An automated hierarchical gait pattern identification tool employing cross‐correlation‐based feature extraction and recurrent neural network based classification , 2009, Expert Syst. J. Knowl. Eng..

[40]  Paul J. M. Havinga,et al.  Activity Recognition Using Inertial Sensing for Healthcare, Wellbeing and Sports Applications: A Survey , 2010, ARCS Workshops.

[41]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[42]  Alan Bundy,et al.  Dynamic Time Warping , 1984 .

[43]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[44]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Lei Chen,et al.  Robust and fast similarity search for moving object trajectories , 2005, SIGMOD '05.

[46]  Sebastian Madgwick,et al.  Estimation of IMU and MARG orientation using a gradient descent algorithm , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[47]  Bernt Schiele,et al.  Dead reckoning from the pocket - An experimental study , 2010, 2010 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[48]  Jie Yang,et al.  User Verification Leveraging Gait Recognition for Smartphone Enabled Mobile Healthcare Systems , 2015, IEEE Transactions on Mobile Computing.

[49]  Qian Wang,et al.  Deep Learning-Based Gait Recognition Using Smartphones in the Wild , 2018, IEEE Transactions on Information Forensics and Security.

[50]  P. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[51]  Myeong-jin Lee,et al.  Step Detection Robust against the Dynamics of Smartphones , 2015, Sensors.

[52]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[53]  Valérie Renaudin,et al.  Step Length Estimation Using Handheld Inertial Sensors , 2012, Sensors.

[54]  D. Joanes,et al.  Comparing measures of sample skewness and kurtosis , 1998 .

[55]  Bing-Yu Sun,et al.  A Study on the Dynamic Time Warping in Kernel Machines , 2007, 2007 Third International IEEE Conference on Signal-Image Technologies and Internet-Based System.

[56]  Mohinder S. Grewal,et al.  Global Positioning Systems, Inertial Navigation, and Integration , 2000 .

[57]  Simon Haykin,et al.  Generalized support vector machines , 1999, ESANN.

[58]  Merryn J Mathie,et al.  Accelerometry: providing an integrated, practical method for long-term, ambulatory monitoring of human movement , 2004, Physiological measurement.

[59]  Roberto Iglesias,et al.  Robust Step Counting for Inertial Navigation with Mobile Phones , 2018, Sensors.

[60]  Sebastian O. H. Madgwick,et al.  An efficient orientation filter for inertial and inertial / magnetic sensor arrays , 2010 .

[61]  Alberto Olivares,et al.  Detection of (In)activity Periods in Human Body Motion Using Inertial Sensors: A Comparative Study , 2012, Sensors.

[62]  Sergio Escalera,et al.  Support vector machines with time series distance kernels for action classification , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[63]  Christoph F. Eick,et al.  K-medoid-style Clustering Algorithms for Supervised Summary Generation , 2004, IC-AI.

[64]  Ms. Najme Zehra Naqvi Step Counting Using Smartphone-Based Accelerometer , 2012 .

[65]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[66]  Andreas Savvides,et al.  PEM-ID: Identifying people by gait-matching using cameras and wearable accelerometers , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[67]  Jeen-Shing Wang,et al.  Using acceleration measurements for activity recognition: An effective learning algorithm for constructing neural classifiers , 2008, Pattern Recognit. Lett..

[68]  Peilin Liu,et al.  An improved indoor localization method using smartphone inertial sensors , 2013, International Conference on Indoor Positioning and Indoor Navigation.

[69]  Tomoya Ishikawa,et al.  A method of pedestrian dead reckoning using action recognition , 2010, IEEE/ION Position, Location and Navigation Symposium.

[70]  Xiaogang Wang,et al.  A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[71]  Weihua Sheng,et al.  Recognizing human daily activity using a single inertial sensor , 2010, 2010 8th World Congress on Intelligent Control and Automation.

[72]  Valérie Renaudin,et al.  Motion Mode Recognition and Step Detection Algorithms for Mobile Phone Users , 2013, Sensors.