Gesture Profile for Web Services: An Event-Driven Architecture to Support Gestural Interfaces for Smart Environments

Gestural interfaces have lately become extremely popular due to the introduction on the market of low-cost acquisition devices such as iPhone, Wii, and Kinect. Such devices allow practitioners to design, experiment, and evaluate novel interfaces and interactions for new smart environments. However, gesture recognition algorithms are currently the appanage of machine learning experts which sometimes leaves AmI practitioners dealing with complex pattern recognition techniques instead of focusing on prototyping ambient interactions. To address this problem, we propose GPWS (Gesture Profile for Web Services), a service-oriented architecture (SOA) designed to assist implementation of gestural interfaces. By providing gesture recognition as a web service, we leverage easy and fast adoption of gestural interfaces for various platforms and environments through simple service discovery and composition mechanisms. We discuss two GPWS designs based on SOA 1.0 and SOA 2.0 standards, analyze their performance, and demonstrate GPWS for a gesture-controlled smart home application.

[1]  Anthony Rowe,et al.  eWatch: a wearable sensor and notification platform , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[2]  David Luckham,et al.  The power of events - an introduction to complex event processing in distributed enterprise systems , 2002, RuleML.

[3]  Geraldine Fitzpatrick,et al.  Making the Wii at Home: Game Play by Older People in Sheltered Housing , 2010, USAB.

[4]  Fariba Sadri,et al.  Ambient intelligence: A survey , 2011, CSUR.

[5]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[6]  Wei-Tek Tsai,et al.  Gesture ontology for informing Service-oriented Architecture , 2012, 2012 IEEE International Symposium on Intelligent Control.

[7]  Rémi Ronfard,et al.  A survey of vision-based methods for action representation, segmentation and recognition , 2011, Comput. Vis. Image Underst..

[8]  Sebastian Möller,et al.  I'm home: Defining and evaluating a gesture set for smart-home control , 2011, Int. J. Hum. Comput. Stud..

[9]  Manfred Tscheligi,et al.  Trends in the living room and beyond: results from ethnographic studies using creative and playful probing , 2008, CIE.

[10]  Yang Li,et al.  User-defined motion gestures for mobile interaction , 2011, CHI.

[11]  Emile H. L. Aarts,et al.  Ambient Intelligence 2.0: Towards Synergetic Prosperity , 2009, J. Ambient Intell. Smart Environ..

[12]  I. Scott MacKenzie,et al.  An empirical comparison of "wiimote" gun attachments for pointing tasks , 2009, EICS '09.

[13]  Ronald Poppe,et al.  Vision-based human motion analysis: An overview , 2007, Comput. Vis. Image Underst..

[14]  Abu Saleh Md. Mahfujur Rahman,et al.  Motion-path based gesture interaction with smart home services , 2009, ACM Multimedia.

[15]  T. C. Nicholas Graham,et al.  Companion Proceedings of the 12th ACM SIGCHI Symposium on Engineering Interactive Computing Systems , 2020 .

[16]  Zhen Wang,et al.  uWave: Accelerometer-based Personalized Gesture Recognition and Its Applications , 2009, PerCom.

[17]  Niels Henze,et al.  Gesture recognition with a Wii controller , 2008, TEI.

[18]  Maojun Zhang,et al.  An application oriented and shape feature based multi-touch gesture description and recognition method , 2011, Multimedia Tools and Applications.

[19]  Meredith Ringel Morris,et al.  User-defined gestures for surface computing , 2009, CHI.

[20]  Fulvio Corno,et al.  What would you ask to your home if it were intelligent? Exploring user expectations about next-generation homes , 2011, J. Ambient Intell. Smart Environ..

[21]  Joseph J. LaViola,et al.  Breaking the status quo: Improving 3D gesture recognition with spatially convenient input devices , 2010, 2010 IEEE Virtual Reality Conference (VR).

[22]  Radu-Daniel Vatavu,et al.  Nomadic gestures: A technique for reusing gesture commands for frequent ambient interactions , 2012, J. Ambient Intell. Smart Environ..

[23]  Yang Li,et al.  Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes , 2007, UIST.

[24]  Norbert Reithinger,et al.  SmartWeb Handheld - Multimodal Interaction with Ontological Knowledge Bases and Semantic Web Services , 2007, Artifical Intelligence for Human Computing.

[25]  Zhaohui Wu,et al.  GeeAir: a universal multimodal remote control device for home appliances , 2010, Personal and Ubiquitous Computing.

[26]  Debashis Ghosh,et al.  Script Recognition—A Review , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Galen Panger.  Kinect in the kitchen: testing depth camera interactions in practical home environments , 2012, CHI EA '12.

[28]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[29]  Wei-Tek Tsai,et al.  Service-oriented smart home applications: composition, code generation, deployment, and execution , 2011, Service Oriented Computing and Applications.

[30]  Boris E. R. de Ruyter,et al.  New research perspectives on Ambient Intelligence , 2009, J. Ambient Intell. Smart Environ..

[31]  Paul Fergus,et al.  Dynamic service composition in home appliance networks , 2006, Multimedia Tools and Applications.

[32]  Alex Pentland,et al.  Artifical Intelligence for Human Computing, ICMI 2006 and IJCAI 2007 International Workshops, Banff, Canada, November 3, 2006, Hyderabad, India, January 6, 2007, Revised Seleced and Invited Papers , 2007, Artifical Intelligence for Human Computing.

[33]  Dave A. Thomas,et al.  The future of SOA: what worked, what didn't, and where is it going from here? , 2007, OOPSLA '07.

[34]  Filip De Turck,et al.  WS-Gesture, a gesture-based state-aware control framework , 2010, 2010 IEEE International Conference on Service-Oriented Computing and Applications (SOCA).

[35]  C. Creider.  Hand and Mind: What Gestures Reveal about Thought , 1994 .

[36]  Basilio Sierra,et al.  Layered architecture for real time sign recognition: Hand gesture and movement , 2010, Eng. Appl. Artif. Intell..

[37]  Malcolm Munro,et al.  Understanding service-oriented software , 2004, IEEE Software.

[38]  K. M. Lee,et al.  Games for a better life: effects of playing Wii games on the well-being of seniors in a long-term care facility , 2009 .

[39]  Wei-Tek Tsai,et al.  Ontology-Based Smart Home Solution and Service Composition , 2009, 2009 International Conference on Embedded Software and Systems.

[40]  Jorge Parra,et al.  A survey of dynamic service composition approaches for ambient systems , 2008, SOMITAS '08.

[41]  Andreas Holzinger,et al.  HCI in Work and Learning, Life and Leisure - 6th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering, USAB 2010, Klagenfurt, Austria, November 4-5, 2010. Proceedings , 2010, USAB.

[42]  Johnny Chung Lee,et al.  Hacking the Nintendo Wii Remote , 2008, IEEE Pervasive Computing.

[43]  Timothy W. Finin,et al.  Service Composition for Mobile Environments , 2005, Mob. Networks Appl..

[44]  Radu-Daniel Vatavu.  Point & click mediated interactions for large home entertainment displays , 2010, Multimedia Tools and Applications.

[45]  Lisa Anthony,et al.  A lightweight multistroke recognizer for user interface prototypes , 2010, Graphics Interface.

[46]  Joseph J. LaViola,et al.  Exploring 3D gestural interfaces for music creation in video games , 2009, FDG.