Vision-Based Markerless Gaming Interface

The paper proposes a novel human machine interface for gaming applications based on computer vision. The key idea is to allow the user to interact with the game by simply moving a hand-held consumer grade camera. Detection of natural features in the incoming video stream avoids instrumenting the scene with optical markers while preserving real-time computation and accuracy. The paper presents also a prototype videogame developed as proof-of-concept of our camera-based gaming interface. Thanks to recent advances in real-time extraction and matching of natural features from images on mobile platforms, our proposal holds the potential to enable a new generation of camera-controlled videogames for hand-held mobile devices.

[1]  D. B. Davis,et al.  Intel Corp. , 1993 .

[2]  Bruce H. Thomas,et al.  ARQuake: an outdoor/indoor augmented reality first person application , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[3]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[4]  Peng Lu,et al.  Navigation in 3D game by Markov model based head pose estimating , 2004, Third International Conference on Image and Graphics (ICIG'04).

[5]  Dieter Schmalstieg,et al.  Pose tracking from natural features on mobile phones , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[6]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[7]  Suya You,et al.  A video-based augmented reality golf simulator , 2000, MM 2000.

[8]  Michael Harville,et al.  Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera , 2004, CVPR 2004.

[9]  Dieter Schmalstieg,et al.  Towards Massively Multi-user Augmented Reality on Handheld Devices , 2005, Pervasive.

[10]  Sean White,et al.  Developing an augmented reality racing game , 2008, INTETAIN '08.

[11]  Hyun Seung Yang,et al.  Ghost Hunter: A Handheld Augmented Reality Game System with Dynamic Environment , 2007, ICEC.

[12]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[13]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[14]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[15]  Andrew W. Fitzgibbon,et al.  Markerless tracking using planar structures in the scene , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[16]  Mark Fiala,et al.  ARTag, a fiducial marker system using digital techniques , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  Yufeng Chen,et al.  A Vision Based Game Control Method , 2005, ICCV-HCI.

[18]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[19]  Luigi di Stefano,et al.  SVR-Based Jitter Reduction for Markerless Augmented Reality , 2009, ICIAP.

[20]  Kazuo Kyuma,et al.  Computer vision for computer games , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.