A Quaternion-Based Motion Tracking and Gesture Recognition System Using Wireless Inertial Sensors

This work examines the development of a unified motion tracking and gesture recognition system that functions through worn inertial sensors. The system is comprised of a total of ten sensors and uses their quaternion output to map the player's motions to an onscreen character. To demonstrate the system's capabilities, a simple virtual reality game was created. A hierarchical skeletal model was implemented that allows players to navigate the virtual world without the need of a hand-held controller. In addition to motion tracking, the system was tested for its potential for gesture recognition. Despite the widespread use of Hidden Markov Models, our modified Markov Chain algorithm obtained higher average recognition accuracies at 95% and faster computation times. Combining motion tracking and dynamic gesture recognition into a single unified system is unique in the literature and comes at a time when virtual reality and wearable computing are emerging in the marketplace.

[1]  Changchun Liu,et al.  Dynamic Difficulty Adjustment in Computer Games Through Real-Time Anxiety-Based Affective Feedback , 2009, Int. J. Hum. Comput. Interact..

[2]  Kongqiao Wang,et al.  A Framework for Hand Gesture Recognition Based on Accelerometer and EMG Sensors , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[3]  Susannah Hollister.  Dance Dance Revolution , 2012 .

[4]  Brian McDonald,et al.  Intelligent Biofeedback using an Immersive Competitive Environment , 2001 .

[5]  James Diebel,et al.  Representing Attitude : Euler Angles , Unit Quaternions , and Rotation Vectors , 2006 .

[6]  Rodrigo de Oliveira,et al.  TripleBeat: enhancing exercise performance with persuasion , 2008, Mobile HCI.

[7]  Gernot R. Müller-Putz,et al.  Game-like Training to Learn Single Switch Operated Neuroprosthetic Control , 2007 .

[8]  Paul Fergus,et al.  Body Area Networks for Movement Analysis in Physiotherapy Treatments , 2010, 2010 IEEE 24th International Conference on Advanced Information Networking and Applications Workshops.

[9]  Alan J. Dix,et al.  Affective Videogames and Modes of Affective Gaming: Assist Me, Challenge Me, Emote Me (ACE) , 2005, DiGRA Conference.

[10]  Graham Bruce Baradoy,et al.  A Physiological Feedback Controlled Exercise Video Game , 2012 .

[11]  Marko Turpeinen,et al.  The influence of implicit and explicit biofeedback in first-person shooter games , 2010, CHI.

[12]  Erik Champion,et al.  Please Biofeed the Zombies: Enhancing the Gameplay and Display of a Horror Game Using Biofeedback , 2007, DiGRA Conference.

[13]  Reed W Hoyt.  SPARNET - Spartan Data Network for Real-Time Physiological Status Monitoring , 2008 .

[14]  Frank J. Lee,et al.  Wiizards: 3D gesture recognition for game play input , 2007, Future Play.

[15]  John A. Stankovic,et al.  ALARM-NET: Wireless Sensor Networks for Assisted-Living and Residential Monitoring , 2006 .

[16]  Biing-Hwang Juang,et al.  Feature Processing and Modeling for 6D Motion Gesture Recognition , 2013, IEEE Transactions on Multimedia.

[17]  Anton Nijholt,et al.  Turning Shortcomings into Challenges: Brain-Computer Interfaces for Games , 2009, INTETAIN.

[18]  Anthony Whitehead,et al.  Sensor networks as video game input devices , 2007, Future Play.

[19]  Sunwook Kim,et al.  Performance evaluation of a wearable inertial motion capture system for capturing physical exposures during manual material handling tasks , 2013, Ergonomics.

[20]  Junichi Hoshino,et al.  A fitness game reflecting heart rate , 2006, ACE '06.

[21]  Rong Zhu,et al.  A real-time articulated human motion tracking using tri-axis inertial/magnetic sensors package , 2004, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  —Ralph Waldo Emerson.  Glass , 2007, Bulletin of the Detroit Institute of Arts.

[23]  Sethuraman Panchanathan,et al.  Body Sensor Networks for Activity and Gesture Recognition , 2014 .

[24]  Robert B. McGhee,et al.  Design, Implementation, and Experimental Results of a Quaternion-Based Kalman Filter for Human Body Motion Tracking , 2005, IEEE Transactions on Robotics.

[25]  Jeff A. Johnson,et al.  The Xerox Star: a retrospective , 1989, Computer.

[26]  Matt Welsh,et al.  Sensor networks for medical care , 2005, SenSys '05.

[27]  Surapa Thiemjarus,et al.  Dogsperate Escape: A Demonstration of Real-Time BSN-Based Game Control with e-AR Sensor , 2010, KICSS.

[28]  Anthony Whitehead.  Gesture Recognition with Accelerometers for Game Controllers, Phones and Wearables , 2014 .

[29]  Aravind Kailas.  Basic human motion tracking using a pair of gyro + accelerometer MEMS devices , 2012, 2012 IEEE 14th International Conference on e-Health Networking, Applications and Services (Healthcom).

[30]  Luca Chittaro,et al.  Smarter Phones for Healthier Lifestyles: An Adaptive Fitness Game , 2010, IEEE Pervasive Computing.

[31]  Luca Benini,et al.  MOCA: A Low-Power, Low-Cost Motion Capture System Based on Integrated Accelerometers , 2007, Adv. Multim..

[32]  Majid Sarrafzadeh,et al.  Near-Realistic Motion Video Games with Enforced Activity , 2012, 2012 Ninth International Conference on Wearable and Implantable Body Sensor Networks.

[33]  Jani Mäntyjärvi,et al.  Accelerometer-based gesture control for a design environment , 2006, Personal and Ubiquitous Computing.

[34]  Lawrence Wai-Choong Wong,et al.  Ubiquitous Human Upper-Limb Motion Estimation using Wearable Sensors , 2011, IEEE Transactions on Information Technology in Biomedicine.

[35]  Salvatore Sessa,et al.  Towards Miniaturization of a MEMS-Based Wearable Motion Capture System , 2011, IEEE Transactions on Industrial Electronics.

[36]  J. Pineda,et al.  Learning to control brain rhythms: making a brain-computer interface possible , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  M. Arditti.  Unity , 1957, Journal of the Irish Medical Association.

[38]  Jing Yang,et al.  A novel hand gesture input device based on inertial sensing technique , 2004, 30th Annual Conference of IEEE Industrial Electronics Society, 2004. IECON 2004.

[39]  Shengli Zhou,et al.  Hand-written character recognition using MEMS motion sensing technology , 2008, 2008 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[40]  Michael L. Littman,et al.  Activity Recognition from Accelerometer Data , 2005, AAAI.

[41]  Anthony Whitehead,et al.  Pose presentation for a dance-based massively multiplayer online exergame , 2011, Entertain. Comput..

[42]  P H Veltink,et al.  Ambulatory measurement of arm orientation. , 2007, Journal of biomechanics.

[43]  D. Vojta,et al.  Effects of a pediatric weight management program with and without active video games a randomized trial. , 2014, JAMA pediatrics.

[44]  Peter H. Veltink,et al.  Measuring orientation of human body segments using miniature gyroscopes and accelerometers , 2005, Medical and Biological Engineering and Computing.

[45]  Mark J. Buller,et al.  Combat medical informatics: present and future , 2002, AMIA.

[46]  S. Gordon,et al.  Waging war on modern chronic diseases: primary prevention through exercise biology , 2000, Journal of applied physiology.

[47]  Jwu-Sheng Hu,et al.  Dancing game by digital textile sensor, accelerometer and gyroscope , 2011, 2011 IEEE International Games Innovation Conference (IGIC).

[48]  Magnus Moar,et al.  ‘Ere Be Dragons: heartfelt gaming , 2006, Digit. Creativity.

[49]  Emiko Charbonneau,et al.  Poster: RealDance: An exploration of 3D spatial interfaces for dancing games , 2009, 2009 IEEE Symposium on 3D User Interfaces.

[50]  Jong Hyuk Park,et al.  Adaptive Body Posture Analysis for Elderly-Falling Detection with Multisensors , 2010, IEEE Intelligent Systems.

[51]  R. Mukundan.  Quaternions : From Classical Mechanics to Computer Graphics , and Beyond , 2002 .

[52]  Andrea Vitali,et al.  Development of a wireless low-power multi-sensor network for motion tracking applications , 2013, 2013 IEEE International Conference on Body Sensor Networks.

[53]  Zhipei Huang,et al.  Adaptive Information Fusion for Human Upper Limb Movement Estimation , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[54]  Sara Ilstedt Hjelm,et al.  Brainball - using brain activity for cool competition , 2000 .

[55]  J. L. Roux.  An Introduction to the Kalman Filter , 2003 .

[56]  Ingrid Moerman,et al.  A Comprehensive Survey of Wireless Body Area Networks , 2012, Journal of Medical Systems.

[57]  Joseph J. LaViola,et al.  3D Gestural Interaction: The State of the Field , 2013 .

[58]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[59]  日向 俊二.  Kinect for Windowsアプリを作ろう , 2012 .

[60]  Anthony Whitehead,et al.  Homogeneous accelerometer-based sensor networks for game interaction , 2011, CIE.

[61]  Greg Welch,et al.  Motion Tracking: No Silver Bullet, but a Respectable Arsenal , 2002, IEEE Computer Graphics and Applications.

[62]  Rabab K Ward,et al.  A survey of signal processing algorithms in brain–computer interfaces based on electrical brain signals , 2007, Journal of neural engineering.

[63]  Alan F. Smeaton,et al.  Aggregating multiple body sensors for analysis in sports , 2008 .

[64]  Brendan Iribe.  Oculus Rift를 이용한 체감형 게임 구현 , 2014 .

[65]  Jorge Lobo,et al.  Distributed Accelerometers for Gesture Recognition and Visualization , 2011, DoCEIS.

[66]  Divya Sharma,et al.  Body area networks: A survey , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[67]  Jani Mäntyjärvi,et al.  Enabling fast and effortless customisation in accelerometer based gesture interaction , 2004, MUM '04.

[68]  Regan L. Mandryk,et al.  Using psychophysiological techniques to measure user experience with entertainment technologies , 2006, Behav. Inf. Technol..

[69]  Yu-Chee Tseng,et al.  Multi-screen cyber-physical video game: An integration with body-area inertial sensor networks , 2010, 2010 8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops).

[70]  Regan L. Mandryk,et al.  Biofeedback game design: using direct and indirect physiological control to enhance game interaction , 2011, CHI.