Low-Latency Filtering of Kinect Skeleton Data for Video Game Control

The Kinect sensor was originally designed for use as a controller for video games, however the noise and latency in the output of its skeleton tracking system negatively impact user experience. This paper compares four different filter-based approaches to mitigating that noise and latency in a simple video game: the Kinect SDK's built-in Holt double exponential smoothing filter, an averaging filter, a Kalman filter with a constant-value model, and a Kalman filter with a Wiener Process Acceleration (WPA) model. The averaging filter and constant-value Kalman filter both have good smoothing performance but introduce relatively large amounts of latency, while the others have good performance in both measures. The WPA Kalman filter was found to have the best overall performance.

[1]  Prajakta S. Kalekar.  Time series Forecasting using Holt-Winters Exponential Smoothing , 2004 .

[2]  Achmad Rizal,et al.  Proportional derivative control based robot arm system using Microsoft Kinect , 2013, 2013 International Conference on Robotics, Biomimetics, Intelligent Computational Systems.

[3]  M. Ferraro,et al.  A novel bio-kinematic encoder for human exercise representation and decomposition - Part 2: Robustness and optimisation , 2013, 2013 International Conference on Control, Automation and Information Sciences (ICCAIS).

[4]  Erik Blasch,et al.  Comparison of three approximate kinematic models for space object tracking , 2013, Proceedings of the 16th International Conference on Information Fusion.

[5]  Markus Santoso,et al.  Immersive Driving Car Simulation for Children Using Natural User Interface Controller , 2013, 2013 International Symposium on Ubiquitous Virtual Reality.

[6]  Luiz Velho,et al.  Kinect and RGBD Images: Challenges and Applications , 2012, 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images Tutorials.

[7]  Martin Jägersand,et al.  SEPO: Selecting by pointing as an intuitive human-robot command interface , 2013, 2013 IEEE International Conference on Robotics and Automation.

[8]  P. S. Archambault,et al.  Evaluation of Kinect skeletal tracking in a virtual reality rehabilitation system for upper limb hemiparesis , 2013, 2013 International Conference on Virtual Rehabilitation (ICVR).

[9]  Sridhar Mahadevan,et al.  Switching kalman filters for prediction and tracking in an adaptive meteorological sensing network , 2005, 2005 Second Annual IEEE Communications Society Conference on Sensor and Ad Hoc Communications and Networks, 2005. IEEE SECON 2005..

[10]  Abdul Muis,et al.  Development of whole body motion imitation in humanoid robot , 2013, 2013 International Conference on QiR.

[11]  Dagmar Sternad,et al.  Sensitivity of Smoothness Measures to Movement Duration, Amplitude, and Arrests , 2009, Journal of motor behavior.

[12]  Etienne Burdet,et al.  A Robust and Sensitive Metric for Quantifying Movement Smoothness , 2012, IEEE Transactions on Biomedical Engineering.

[13]  Thijs Vandenryt,et al.  Waving at the Heart: Implementation of a Kinect-based real-time interactive control system for viewing cineangiogram loops during cardiac catheterization procedures , 2013, Computing in Cardiology 2013.

[14]  Mark Claypool,et al.  The effects of loss and latency on user performance in unreal tournament 2003® , 2004, NetGames '04.

[15]  David Webster,et al.  Experimental evaluation of Microsoft Kinect's accuracy and capture rate for stroke rehabilitation applications , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[16]  Lars C. Wolf,et al.  On the impact of delay on real-time multiplayer games , 2002, NOSSDAV '02.

[17]  Chung-Hung Hsieh,et al.  A Kinect-Based System for Physical Rehabilitation: Utilizing Tai Chi Exercises to Improve Movement Disorders in Patients with Balance Ability , 2013, 2013 7th Asia Modelling Symposium.

[18]  F. Duan,et al.  A study of the human-robot synchronous control system based on skeletal tracking technology , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[19]  Ling Guan,et al.  Quantifying and recognizing human movement patterns from monocular video Images-part I: a new framework for modeling human motion , 2004, IEEE Transactions on Circuits and Systems for Video Technology.