Adaptive and Personalized Gesture Recognition Using Textile Capacitive Sensor Arrays

Upper extremity mobility impairment is a common sequel of Spinal Cord Injury (SCI), brain injury, strokes, and degenerative diseases such as Guillain-Barre and ALS. Existing assistive technology solutions that provide access as user input devices are intrusive and expensive, and require physical contact that can have deleterious effects such as skin friction injury for paralyzed users who have reduced skin sensitivity. To address this problem, in this paper, we present the design, implementation, and evaluation of a non-contact proximity gesture recognition system using fabric capacitive sensor arrays. The fabric sensors are lightweight, flexible, and can be easily integrated into items of quotidian use such as clothing, bed sheets, and pillow covers. Our gesture recognition algorithm builds on two known classification techniques, Hidden Markov Model and Dynamic Time Warping to convert raw capacitance values to alphanumeric gestures. Our system is personalized to the user, allowing personalized selection of gesture sets and definition of gesture patterns in accordance with their capabilities. Our system adapts to changes in sensor configuration and orientation with minimal user training and intervention. We have evaluated our system in the context of a gesture-driven home automation system on six subjects that includes an individual who has a C6 Spinal Cord injury. We show that our system can recognize gestures of varying complexity with an average accuracy of 99 percent with minimal training.

[1]  Xu Zhou,et al.  A FOOTPRINT TRACKING METHOD FOR GAIT ANALYSIS , 2014 .

[2]  Ying Sun,et al.  Voice-activated environmental control system for persons with disabilities , 2000, Proceedings of the IEEE 26th Annual Northeast Bioengineering Conference (Cat. No.00CH37114).

[3]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[4]  Aaron F. Bobick,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  A. Nelson,et al.  Wearable multi-sensor gesture recognition for paralysis patients , 2013, 2013 IEEE SENSORS.

[6]  Pinkuan Liu,et al.  Conditioning circuit for capacitive position sensor with nano-scale precision based on AC excitation principle , 2011, 2011 IEEE INTERNATIONAL CONFERENCE ON ELECTRO/INFORMATION TECHNOLOGY.

[7]  J. Langlois,et al.  The Epidemiology and Impact of Traumatic Brain Injury: A Brief Overview , 2006, The Journal of head trauma rehabilitation.

[8]  Brad A. Myers,et al.  EdgeWrite: a stylus-based text entry method designed for high accuracy and stability of motion , 2003, UIST '03.

[9]  W Seamone,et al.  An assistive equipment controller for quadriplegics. , 1979, The Johns Hopkins medical journal.

[10]  R Netsell,et al.  Restoration of intelligible speech 13 years post-head injury. , 1992, Brain injury.

[11]  Pavel Ripka,et al.  Modern Sensors Handbook , 2007 .

[12]  Prashan Premaratne,et al.  Consumer electronics control system based on hand gesture moment invariants , 2007 .

[13]  J H Jaffin,et al.  Lightning strike to the head: case report. , 1994, The Journal of trauma.

[14]  Paul Lukowicz,et al.  Gesture spotting with body-worn inertial sensors to detect user activities , 2008, Pattern Recognit..

[15]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[16]  Theodor Groz,et al.  Facts and Figures at a Glance , 2007 .

[17]  Jungwoo Lee,et al.  A programmable mutual capacitance sensing circuit for a large-sized touch panel , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[18]  Wen Gao,et al.  Large vocabulary sign language recognition based on hierarchical decision trees , 2003, ICMI '03.

[19]  Chintan Patel,et al.  Inviz: Low-power personalized gesture recognition using wearable textile capacitive sensor arrays , 2015, 2015 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[20]  Paul Lukowicz,et al.  Active Capacitive Sensing: Exploring a New Wearable Sensing Modality for Activity Recognition , 2010, Pervasive.

[21]  Yifan Wang,et al.  Three dimensional touchless tracking of objects using integrated capacitive sensors , 2012, IEEE Transactions on Consumer Electronics.

[22]  John Hershberger,et al.  An O(nlogn) implementation of the Douglas-Peucker algorithm for line simplification , 1994, SCG '94.

[23]  Ying Wu,et al.  Vision-Based Gesture Recognition: A Review , 1999, Gesture Workshop.

[24]  Jing Yang,et al.  Magic wand: a hand-drawn gesture input device in 3-D space with inertial sensors , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[25]  L. Zhao,et al.  Micro Capacitive Tilt Sensor for Human Body Movement Detection , 2007, BSN.

[26]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[27]  Richard M. Schwartz,et al.  On-line cursive handwriting recognition using speech recognition methods , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[28]  R. Ramadoss,et al.  MEMS-Capacitive Pressure Sensor Fabricated Using Printed-Circuit-Processing Techniques , 2006, IEEE Sensors Journal.

[29]  Joseph A. Paradiso,et al.  An Inertial Measurement Framework for Gesture Recognition and Applications , 2001, Gesture Workshop.

[30]  Se Dong Min,et al.  Noncontact Respiration Rate Measurement System Using an Ultrasonic Proximity Sensor , 2010, IEEE Sensors Journal.

[31]  Chitsung Hong,et al.  Implementation of vertical-integrated dual mode inductive-capacitive proximity sensor , 2012, 2012 IEEE 25th International Conference on Micro Electro Mechanical Systems (MEMS).

[32]  Chitsung Hong,et al.  Implementation of inductive proximity sensor using nanoporous anodic aluminum oxide layer , 2011, 2011 16th International Solid-State Sensors, Actuators and Microsystems Conference.

[33]  Judith M Burnfield,et al.  Patients' experiences with technology during inpatient rehabilitation: opportunities to support independence and therapeutic engagement , 2014, Disability and rehabilitation. Assistive technology.

[34]  Romesh Nagarajah,et al.  A Neural Network Approach to Fluid Quantity Measurement in Dynamic Environments , 2012 .