Distributed Spatiotemporal Gesture Recognition in Sensor Arrays

We present algorithms for gesture recognition using in-network processing in distributed sensor arrays embedded within systems such as tactile input devices, sensing skins for robotic applications, and smart walls. We describe three distributed gesture-recognition algorithms that are designed to function on sensor arrays with minimal computational power, limited memory, limited bandwidth, and possibly unreliable communication. These constraints cause storage of gesture templates within the system and distributed consensus algorithms for recognizing gestures to be difficult. Building up on a chain vector encoding algorithm commonly used for gesture recognition on a central computer, we approach this problem by dividing the gesture dataset between nodes such that each node has access to the complete dataset via its neighbors. Nodes share gesture information among each other, then each node tries to identify the gesture. In order to distribute the computational load among all nodes, we also investigate an alternative algorithm, in which each node that detects a motion will apply a recognition algorithm to part of the input gesture, then share its data with all other motion nodes. Next, we show that a hybrid algorithm that distributes both computation and template storage can address trade-offs between memory and computational efficiency.

[1]  Ying Wu,et al.  Vision-Based Gesture Recognition: A Review , 1999, Gesture Workshop.

[2]  Nikolaus Correll,et al.  Materials that couple sensing, actuation, computation, and communication , 2015, Science.

[3]  Niels Henze,et al.  Gesture recognition with a Wii controller , 2008, TEI.

[4]  Nikolaus Correll,et al.  Bloom Filter-Based Ad Hoc Multicast Communication in Cyber-Physical Systems and Computational Materials , 2012, WASA.

[5]  Bharti Bansal,et al.  Gesture Recognition: A Survey , 2016 .

[6]  Luca Maria Gambardella,et al.  Incremental learning using partial feedback for gesture-based human-swarm interaction , 2012, 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.

[7]  Nikolaus Correll,et al.  Gesture based distributed user interaction system for a reconfigurable self-organizing smart wall , 2014, TEI '14.

[8]  Mark T. Jones,et al.  Communications in Electronic Textile Systems , 2003, Communications in Computing.

[9]  V. Lumelsky,et al.  Sensitive skin , 2000, IEEE Sensors Journal.

[10]  Nikolaus Correll,et al.  Flutter: An Exploration of an Assistive Garment Using Distributed Sensing, Computation and Actuation , 2015, Tangible and Embedded Interaction.

[11]  Claus Bahlmann,et al.  Online handwriting recognition with support vector machines - a kernel approach , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[12]  Aaron F. Bobick,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Joseph A. Paradiso,et al.  Pushpin Computing System Overview: A Platform for Distributed, Embedded, Ubiquitous Sensor Networks , 2002, Pervasive.

[14]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.

[15]  Alex Waibel,et al.  Readings in speech recognition , 1990 .

[16]  V. Michael Bove,et al.  Programming a paintable computer , 2002 .

[17]  Ayoub Al-Hamadi,et al.  A Hidden Markov Model-Based Isolated and Meaningful Hand Gesture Recognition , 2008 .

[18]  Joan Climent,et al.  A Performance Evaluation of HMM and DTW for Gesture Recognition , 2012, CIARP.

[19]  Nikolaus Correll,et al.  A soft, amorphous skin that can sense and localize textures , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[20]  H. DavisR.,et al.  Recognition of handwritten charactersa review , 1986 .

[21]  M. McEvoy,et al.  Thermoplastic variable stiffness composites with embedded, networked sensing, actuation, and control , 2015 .

[22]  Nikolaus Correll,et al.  Establishing Multi-cast Groups in Computational Robotic Materials , 2012, 2012 IEEE International Conference on Green Computing and Communications.

[23]  F BobickAaron,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999 .

[24]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Joseph A. Paradiso,et al.  Sensate Media — Multimodal Electronic Skins as Dense Sensor Networks , 2004 .

[26]  A. Enis Çetin,et al.  Vision-Based Single-Stroke Character Recognition for Wearable Computing , 2001, IEEE Intell. Syst..

[27]  Robert H. Davis,et al.  Recognition of handwritten characters - a review , 1986, Image Vis. Comput..

[28]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[29]  M. Gribaudo,et al.  2002 , 2001, Cell and Tissue Research.