FRAME-LEVEL PROXIMITY AND TOUCH RECOGNITION USING CAPACITIVE SENSING AND SEMI-SUPERVISED SEQUENTIAL MODELING

This paper demonstrates a semi-supervised learning approach to frame-level proximity and touch recognition with machine learning algorithms for sequential modeling. We focus on capacitive sensing, which is employable in low cost embedded devices and provides high sensing capability. We optimize our models to run with minimum complexity to enable the use of state-of-the-art machine learning models in low cost embedded devices. We evaluate two different models, either based on recurrent neural networks (RNN) with gated recurrent units or hidden markov models (HMM). We show that the developed models are capable of a robust proximity and touch recognition invariant to interference factors. However, the RNN model outperforms the HMM model reaching a superior frame-level recognition accuracy of 97.1% on a challenging set of touches containing multiple interference factors like the use of gloves with different materials and invalid touches, where the test persons swiped over the sensor.

[1]  Chintan Patel,et al.  Event-Driven Low-Power Gesture Recognition Using Differential Capacitance , 2016, IEEE Sensors Journal.

[2]  Ivan Poupyrev,et al.  Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects , 2012, CHI.

[3]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[4]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[5]  Alethea L. Blackler,et al.  Intuitive Interaction research – new directions and possible responses , 2016 .

[6]  Eftichios Koutroulis,et al.  A low-cost capacitive sensor for water level monitoring in large-scale storage tanks , 2015, 2015 IEEE International Conference on Industrial Technology (ICIT).

[7]  Carlo Luschi,et al.  Revisiting Small Batch Training for Deep Neural Networks , 2018, ArXiv.

[8]  Hung T. Nguyen,et al.  Human computer interaction using hand gesture , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[9]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[10]  Jeff Heaton,et al.  An empirical analysis of feature engineering for predictive modeling , 2016, SoutheastCon 2016.

[11]  Atika Arshad,et al.  A capacitive proximity sensing scheme for human motion detection , 2017, 2017 IEEE International Instrumentation and Measurement Technology Conference (I2MTC).

[12]  Ivan Poupyrev,et al.  Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum , 2016, UIST.

[13]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[14]  Gyanendra K. Verma,et al.  Human Computer Interaction using Hand Gesture , 2015 .

[15]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[16]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[17]  Yael Edan,et al.  Vision-based hand-gesture applications , 2011, Commun. ACM.

[18]  Alexander J. Smola,et al.  Efficient mini-batch training for stochastic optimization , 2014, KDD.

[19]  Pritish Narayanan,et al.  Deep Learning with Limited Numerical Precision , 2015, ICML.

[20]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Yimin Liu,et al.  Micro Hand Gesture Recognition System Using Ultrasonic Active Sensing , 2017, IEEE Access.

[23]  Hadi Heidari,et al.  Wearable Capacitive-Based Wrist-Worn Gesture Sensing System , 2017, 2017 New Generation of CAS (NGCAS).

[24]  Dong-Hyun Lee,et al.  Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks , 2013 .

[25]  Amos J Storkey,et al.  Machine Learning and Pattern Recognition : Preliminaries Course , 2009 .

[26]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[27]  Bryce Osoinach.  Proximity Capacitive Sensor Technology for Touch Sensing Applications , 2008 .

[28]  Piotr Fryzlewicz,et al.  Wild binary segmentation for multiple change-point detection , 2014, 1411.0858.