Touch-less interactive augmented reality game on vision-based wearable device

Abstract There is an increasing interest in creating pervasive games based on emerging interaction technologies. In order to develop touch-less, interactive and augmented reality games on vision-based wearable device, a touch-less motion interaction technology is designed and evaluated in this work. Users interact with the augmented reality games with dynamic hands/feet gestures in front of the camera, which triggers the interaction event to interact with the virtual object in the scene. Three primitive augmented reality games with eleven dynamic gestures are developed based on the proposed touch-less interaction technology as proof. At last, a comparing evaluation is proposed to demonstrate the social acceptability and usability of the touch-less approach, running on a hybrid wearable framework or with Google Glass, as well as workload assessment, user’s emotions and satisfaction.

[1]  Haibo Li,et al.  Interactive Feet for Mobile Immersive Interaction , 2012 .

[2]  International Conference on Making Sense of Converging Media, Academic MindTrek '13, Tampere, Finland, October 1-4, 2013 , 2013, MindTrek.

[3]  Horst Bischof,et al.  Hough-based tracking of non-rigid objects , 2013, Comput. Vis. Image Underst..

[4]  K. Scherer What are emotions? And how can they be measured? , 2005 .

[5]  CheokAdrian David ACM SIGCHI international conference on advances in computer entertainment technology , 2004 .

[6]  Isabelle M. Demeure,et al.  Gaming on the move: urban experience as a new paradigm for mobile pervasive game design , 2010, Multimedia Systems.

[7]  Zhihan Lv,et al.  Imagining in-air interaction for hemiplegia sufferer , 2015, 2015 International Conference on Virtual Rehabilitation (ICVR).

[8]  Zhihan Lv,et al.  3D seabed modeling and visualization on ubiquitous context , 2014, SIGGRAPH ASIA Posters.

[9]  Zachary Fitz-Walter,et al.  Mystery at the library: encouraging library exploration using a pervasive mobile game , 2012, OZCHI.

[10]  Yong Chen,et al.  WebVR - - Web Virtual Reality Engine Based on P2P network , 2011, J. Networks.

[11]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[12]  Annika Waern,et al.  Pervasive Games: Theory and Design , 2009 .

[13]  Stefan Duffner,et al.  PixelTrack: A Fast Adaptive Algorithm for Tracking Non-rigid Objects , 2013, ICCV.

[14]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[15]  Kah Chan,et al.  Playing in Traffic : pervasive gaming for commuters , 2011 .

[16]  Oskar Juhlin,et al.  Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services , 2011, MobileHCI 2011.

[17]  Joel Ross Pervasive negabehavior games for environmental sustainability , 2011, CHI EA '11.

[18]  Zhihan Lv,et al.  Finger in air: touch-less interaction on smartphone , 2013, MUM.

[19]  Ge Chen,et al.  A framework for multi-dimensional WebGIS based interactive online virtual community , 2012 .

[20]  Fei Guo,et al.  Multiview image rectification algorithm for parallel camera arrays , 2014, J. Electronic Imaging.

[21]  Annika Wærn,et al.  An in-game reporting tool for pervasive games , 2009, Advances in Computer Entertainment Technology.

[22]  Emmanoel Ferreira,et al.  Ectodiegesis As Immersive Effect In Pervasive Games , 2013, MindTrek.

[23]  Yael Edan,et al.  Vision-based hand-gesture applications , 2011, Commun. ACM.

[24]  Zhihan Lv,et al.  Multimodal Hand and Foot Gesture Interaction for Handheld Devices , 2014, TOMM.

[25]  Yi Wang,et al.  SAGA: array storage as a DB with support for structural aggregations , 2014, SSDBM '14.

[26]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[27]  Zhihan Lv,et al.  Multimodal Hand and Foot Gesture Interaction for Handheld Devices , 2013, TOMM.

[28]  Zdenek Kalal,et al.  Tracking-Learning-Detection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Steve Benford,et al.  The multimedia challenges raised by pervasive games , 2005, ACM Multimedia.

[30]  Shan Gao,et al.  3D seabed: 3D modeling and visualization platform for the seabed , 2014, 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[31]  Olaf Zukunft,et al.  FRAP: a framework for pervasive games , 2009, EICS '09.

[32]  Chen Qian,et al.  Realtime and Robust Hand Tracking from Depth , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Duncan Rowland,et al.  Interweaving mobile games with everyday life , 2006, CHI.

[34]  Masayuki Nakajima,et al.  Proceedings of the 8th International Conference on Virtual Reality Continuum and its Applications in Industry , 2009, VRCAI 2009.

[35]  Hirokazu Kato,et al.  Touch-Space: Mixed Reality Game Space Based on Ubiquitous, Tangible, and Social Computing , 2002, Personal and Ubiquitous Computing.

[36]  Jiachen Yang,et al.  Objective Evaluation Criteria for Stereo Camera Shooting Quality Under Different Shooting Parameters and Shooting Distances , 2015, IEEE Sensors Journal.

[37]  Patrick Baudisch,et al.  Lucid touch: a see-through mobile device , 2007, UIST.

[38]  Damianos Gavalas,et al.  Evaluation of Pervasive Games: Recruitment of Qualified Participants Through Preparatory Game Phases , 2014, IoT360.

[39]  Stephen A. Brewster,et al.  Usable gestures for mobile interfaces: evaluating social acceptability , 2010, CHI.

[40]  Haibo Li,et al.  FingerInk: turn your glass into a digital board , 2013, OZCHI.

[41]  Damianos Gavalas,et al.  Pervasive games research: a design aspects-based state of the art report , 2013, PCI '13.

[42]  Jinxing Hu,et al.  Traffic Management and Forecasting System Based on 3D GIS , 2015, 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[43]  Steve Benford,et al.  Bridging the physical and digital in pervasive gaming , 2005, CACM.

[44]  Volker Paelke,et al.  Foot-based mobile interaction with games , 2004, ACE '04.

[45]  Zhihan Lv,et al.  Hand-free motion interaction on Google Glass , 2014, SIGGRAPH ASIA Mobile Graphics and Interactive Applications.

[46]  Peter Fröhlich,et al.  Civic engagement meets pervasive gaming: towards long-term mobile participation , 2014, CHI Extended Abstracts.

[47]  Xiaolong Zhou,et al.  Vision-based hand interaction and its application in pervasive games , 2009, VRCAI '09.

[48]  Jinxing Hu,et al.  Big City 3D Visual Analysis , 2015, Eurographics.

[49]  Zhihan Lv,et al.  Wearable Smartphone: Wearable Hybrid Framework for Hand and Foot Gesture Interaction on Smartphone , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[50]  Mary Beth Rosson,et al.  CHI '07 Extended Abstracts on Human Factors in Computing Systems , 2007, CHI 2007.

[51]  Xingshe Zhou,et al.  Toward a Group-Aware Smartphone Sensing System , 2014, IEEE Pervasive Computing.

[52]  Rafael Ballagas,et al.  REXplorer: a mobile, pervasive spell-casting game for tourists , 2007, CHI Extended Abstracts.

[53]  Annika Wærn,et al.  The art of game-mastering pervasive games , 2008, ACE '08.

[54]  Ragnar Nohre Deformed template matching by the Viterbi algorithm , 1996, Pattern Recognit. Lett..

[55]  Anind K. Dey,et al.  The Group Context Framework: An Extensible Toolkit for Opportunistic Grouping and Collaboration , 2015, CSCW.

[56]  Euripides G. M. Petrakis,et al.  Matching and Retrieval of Distorted and Occluded Shapes Using Dynamic Programming , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Regan L. Mandryk,et al.  Control your game-self: effects of controller type on enjoyment, motivation, and personality in game , 2013, CHI.

[58]  Zhihan Lv,et al.  Serious game based dysphonic rehabilitation tool , 2015, 2015 International Conference on Virtual Rehabilitation (ICVR).

[59]  Frans Mäyrä,et al.  Challenges for pervasive mobile game design: examining players' emotional responses , 2005, ACE '05.

[60]  Andrew Zisserman,et al.  Hand detection using multiple proposals , 2011, BMVC.

[61]  Horst Bischof,et al.  Hough-based tracking of non-rigid objects , 2011, 2011 International Conference on Computer Vision.

[62]  Jinxing Hu,et al.  Reprint: WebVRGIS Based City Bigdata 3D Visualization and Analysis , 2015 .

[63]  Ramin Zabih,et al.  Dynamic Programming and Graph Algorithms in Computer Vision , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Merrilea J. Mayo,et al.  Games for science and engineering education , 2007, CACM.

[65]  Annika Wærn,et al.  The three-sixty illusion: designing for immersion in pervasive games , 2009, CHI.

[66]  Damianos Gavalas,et al.  Blending history and fiction in a pervasive game prototype , 2014, MUM.

[67]  Tanguy Coenen,et al.  MuseUs: Case study of a pervasive cultural heritage serious game , 2013, JOCCH.

[68]  Ke Zhang,et al.  Research and Application of the 3D Virtual Community Based on WEBVR and RIA , 2009, Comput. Inf. Sci..

[69]  Yingchao Ren,et al.  Methodology for geographical data evolution: three-dimensional particle-based real-time snow simulation with remote-sensing data , 2013 .

[70]  Song Zhang,et al.  Fast log-Gabor-based nonlocal means image denoising methods , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[71]  Wolfgang Broll,et al.  Meeting technology challenges of pervasive augmented reality games , 2006, NetGames '06.

[72]  Li Cheng,et al.  Efficient Hand Pose Estimation from a Single Depth Image , 2013, 2013 IEEE International Conference on Computer Vision.

[73]  Bo Kampmann Walther,et al.  Atomic actions -- molecular experience: theory of pervasive gaming , 2005, CIE.

[74]  Stephen A. Brewster,et al.  Foot tapping for mobile interaction , 2010, BCS HCI.

[75]  Haibo Li,et al.  Experiencing real 3D gestural interaction with mobile devices , 2013, Pattern Recognit. Lett..

[76]  Dingde Jiang,et al.  Network coding-based energy-efficient multicast routing algorithm for multi-hop wireless networks , 2015, J. Syst. Softw..

[77]  Jinxing Hu,et al.  WebVRGIS Based City Bigdata 3D Visualization and Analysis , 2015, ArXiv.

[78]  Rod McCall,et al.  TimeWarp: interactive time travel with a mobile mixed reality game , 2008, Mobile HCI.

[79]  Zhihan Lv,et al.  Game On, Science - How Video Game Technology May Help Biologists Tackle Visualization Challenges , 2013, PloS one.

[80]  LvZhihan,et al.  Touch-less interactive augmented reality game on vision-based wearable device , 2015 .

[81]  Feng Su Auditory scene analysis and recognition with LDA topic model , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[82]  Desney S. Tan,et al.  CHI '11 Extended Abstracts on Human Factors in Computing Systems , 2008, CHI 2011.

[83]  Charlotte Magnusson,et al.  Navigating the world and learning to like it: mobility training through a pervasive game , 2011, Mobile HCI.

[84]  Marco Jahn,et al.  Saving energy at work: the design of a pervasive game for office spaces , 2012, MUM.

[85]  Zhihan Lv,et al.  Extending touch-less interaction on vision based wearable device , 2015, 2015 IEEE Virtual Reality (VR).

[86]  Matthieu Chavent,et al.  Advances in Human-Protein Interaction - Interactive and Immersive Molecular Simulations , 2012 .

[87]  Irma Lindt,et al.  A report on the crossmedia game epidemic menace , 2007, CIE.

[88]  Peng Zhang,et al.  A transform domain-based anomaly detection approach to network-wide traffic , 2014, J. Netw. Comput. Appl..

[89]  Pattie Maes,et al.  WUW - wear Ur world: a wearable gestural interface , 2009, CHI Extended Abstracts.

[90]  Zhihan Lv,et al.  A game based assistive tool for rehabilitation of dysphonic patients , 2015, 2015 3rd IEEE VR International Workshop on Virtual and Augmented Assistive Technology (VAAT).

[91]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.