Deep Learning Analysis and Age Prediction from Shoeprints

Human walking and gaits involve several complex body parts and are influenced by personality, mood, social and cultural traits, and aging. These factors are reflected in shoeprints, which in turn can be used to predict age, a problem not systematically addressed using any computational approach. We collected 100,000 shoeprints of subjects ranging from 7 to 80 years old and used the data to develop a deep learning end-to-end model ShoeNet to analyze age-related patterns and predict age. The model integrates various convolutional neural network models together using a skip mechanism to extract age-related features, especially in pressure and abrasion regions from pair-wise shoeprints. The results show that 40.23% of the subjects had prediction errors within 5-years of age and the prediction accuracy for gender classification reached 86.07%. Interestingly, the age-related features mostly reside in the asymmetric differences between left and right shoeprints. The analysis also reveals interesting age-related and gender-related patterns in the pressure distributions on shoeprints; in particular, the pressure forces spread from the middle of the toe toward outside regions over age with gender-specific variations on heel regions. Such statistics provide insight into new methods for forensic investigations, medical studies of gait-pattern disorders, biometrics, and sport studies.

[1]  T. Gibbens Age and Crime , 1965, Mental health.

[2]  Adam Zemla,et al.  LGA: a method for finding 3D similarities in protein structures , 2003, Nucleic Acids Res..

[3]  Girod Alexandre,et al.  Computerized classification of the shoeprints of burglars' soles , 1996 .

[4]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[5]  E. Radin,et al.  Osteoarthrosis. What is known about prevention. , 1987, Clinical orthopaedics and related research.

[6]  R. Waters,et al.  The energy expenditure of normal and pathologic gait. , 1999, Gait & posture.

[7]  H. Stefánsson,et al.  Brain age prediction using deep learning uncovers associated sequence variants , 2019, Nature Communications.

[8]  Nilufar Baghaei,et al.  Learning Wear Patterns on Footwear Outsoles Using Convolutional Neural Networks , 2019, 2019 18th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/13th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE).

[9]  R. D. Blackledge,et al.  The discrimination of two-dimensional military boot impressions based on wear patterns , 2002 .

[10]  Mark S. Nixon,et al.  Automatic extraction and description of human gait models for recognition purposes , 2003, Comput. Vis. Image Underst..

[11]  J. Hughes,et al.  Foot pressure measurement: a review of clinical findings. , 1986, Journal of biomedical engineering.

[12]  Shaoguo Wen,et al.  Shoe-Print Image Retrieval With Multi-Part Weighted CNN , 2019, IEEE Access.

[13]  Regina Katzenschlager,et al.  Gait disorders in adults and the elderly , 2016, Wiener klinische Wochenschrift.

[14]  Yang Zhang,et al.  Adapting Convolutional Neural Networks on the Shoeprint Retrieval for Forensic Use , 2017, CCBR.

[15]  Scott Krig,et al.  Interest Point Detector and Feature Descriptor Survey , 2014 .

[16]  Imad Rida,et al.  Towards Human Body-Part Learning for Model-Free Gait Recognition , 2019, ArXiv.

[17]  Yasushi Makihara,et al.  Gait Analysis of Gender and Age Using a Large-Scale Multi-view Gait Database , 2010, ACCV.

[18]  K. Khan,et al.  Utility of the mean cumulative function in the analysis of fall events. , 2007, The journals of gerontology. Series A, Biological sciences and medical sciences.

[19]  Jules-Raymond Tapamo,et al.  Age estimation via face images: a survey , 2018, EURASIP Journal on Image and Video Processing.

[20]  Junjian Cui,et al.  Robust Shoeprint Retrieval Method Based on Local‐to‐Global Feature Matching for Real Crime Scenes , 2018, Journal of forensic sciences.

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Richard W. Bohannon,et al.  Normal walking speed: a descriptive meta-analysis. , 2011, Physiotherapy.

[23]  Sargur N. Srihari,et al.  Footwear Print Retrieval System for Real Crime Scene Marks , 2010, ICWF.

[24]  M. Kosinski,et al.  Deep Neural Networks Are More Accurate Than Humans at Detecting Sexual Orientation From Facial Images , 2018, Journal of personality and social psychology.

[25]  Loris Nanni,et al.  Handcrafted vs. non-handcrafted features for computer vision classification , 2017, Pattern Recognit..

[26]  Patrick A. Fitzhorn,et al.  A methodology for near-optimal computational superimposition of two-dimensional digital facial photographs and three-dimensional cranial surface meshes. , 1991, Journal of forensic sciences.

[27]  Derya Atamtürk Estimation of sex from the dimensions of foot, footprints, and shoe. , 2010 .

[28]  William J. Bodziak,et al.  Footwear Impression Evidence: Detection, Recovery and Examination , 1999 .

[29]  A. B. Drought,et al.  WALKING PATTERNS OF NORMAL MEN. , 1964, The Journal of bone and joint surgery. American volume.

[30]  Zeno Geradts,et al.  The image-database REBEZO for shoeprints with developments on automatic classification of shoe outsole designs , 1996 .

[31]  N. Alexander Gait Disorders in Older Adults , 1996, Journal of the American Geriatrics Society.

[32]  A. Turgut,et al.  Stature and sex estimate using foot and shoe dimensions. , 2005, Forensic science international.

[33]  Hazem Refai,et al.  Predicting Age From Brain EEG Signals—A Machine Learning Approach , 2018, Front. Aging Neurosci..

[34]  Tapio Pahikkala,et al.  Measuring Player Retention and Monetization Using the Mean Cumulative Function , 2017, IEEE Transactions on Games.

[35]  Yan Xu,et al.  Human Age Prediction Based on DNA Methylation Using a Gradient Boosting Regressor , 2018, Genes.

[36]  Hedieh Sajedi,et al.  Age prediction based on brain MRI images using feature learning , 2017, 2017 IEEE 15th International Symposium on Intelligent Systems and Informatics (SISY).

[37]  M. Samson,et al.  Differences in gait parameters at a preferred walking speed in healthy subjects due to age, height and body weight , 2001, Aging.

[38]  Jürgen Schmidhuber,et al.  Highway Networks , 2015, ArXiv.

[39]  Izzet Duyar,et al.  Age‐Related Factors in the Relationship Between Foot Measurements and Living Stature and Body Weight , 2008, Journal of forensic sciences.

[40]  B. E. Maki,et al.  Gait Changes in Older Adults: Predictors of Falls or Indicators of Fear? , 1997, Journal of the American Geriatrics Society.

[41]  D. Pacheco Bautista,et al.  COMPUTATIONAL INTELLIGENCE FOR SHOEPRINT RECOGNITION , 2019 .

[42]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[43]  W F Rowe Interpretation of shoe wear patterns in a personal injury case. , 1981, Journal of forensic sciences.

[44]  Pengfei Shi,et al.  Computerized Matching of Shoeprints Based on Sole Pattern , 2008, IWCF.

[45]  L. M. Marado,et al.  Biological Profile Estimation Based on Footprints and Shoeprints from Bracara Augusta Figlinae (Brick Workshops) , 2018 .

[46]  Samir Kumar Bandyopadhyay,et al.  Crime scene reconstruction-Sex prediction from blood stained foot sole impressions. , 2017, Forensic science international.

[47]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[48]  L. Alcock,et al.  Alterations in gait speed and age do not fully explain the changes in gait mechanics associated with healthy older women. , 2013, Gait & posture.

[49]  J E Buikstra,et al.  Linear models for the prediction of stature from foot and boot dimensions. , 1992, Journal of forensic sciences.

[50]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  S. Hutchins,et al.  The effect of aging on gait parameters in able-bodied older subjects: a literature review , 2016, Aging Clinical and Experimental Research.

[52]  Sambit Bakshi,et al.  A comprehensive overview of feature representation for biometric recognition , 2018, Multimedia Tools and Applications.

[53]  L. Ferrucci,et al.  Skeletal muscle mitochondrial energetics are associated with maximal aerobic capacity and walking speed in older adults. , 2013, The journals of gerontology. Series A, Biological sciences and medical sciences.

[54]  S. Studenski,et al.  Gait speed and survival in older adults. , 2011, JAMA.

[55]  Hugo Proença,et al.  Forensic Shoe-print Identification: A Brief Survey , 2019, ArXiv.

[56]  L. Glass Moiré Effect from Random Dots , 1969, Nature.

[57]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Paul T. Jayaprakash,et al.  Skull-Photo Superimposition: A Remedy to the Problem of Unidentified Dead in Malaysia , 2010 .

[59]  Keiichi Abe,et al.  Topological structural analysis of digitized binary images by border following , 1985, Comput. Vis. Graph. Image Process..

[60]  M. Whittle,et al.  Generation and attenuation of transient impulsive forces beneath the foot: a review. , 1999, Gait & posture.

[61]  Charless C. Fowlkes,et al.  Cross-Domain Forensic Shoeprint Matching , 2017 .

[62]  E. A. Okubike,et al.  Stature Prediction using Shoe Print Dimensions of an Adult Nigerian Population , 2018 .

[63]  M. P. Murray Gait as a total pattern of movement. , 1967, American journal of physical medicine.

[64]  W A Aulsebrook,et al.  Superimposition and reconstruction in forensic facial identification: a survey. , 1995, Forensic science international.

[65]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).