A novel gesture-based interface for a VR simulation: Re-discovering Vrouw Maria

In this paper, we present the design and the development of a robust, low-cost and easy to implement system for a gesture-based interface that enables visitors to navigate in 3D in a virtual reality environment. By using a state-of-the-art 3D motion sensing device, Microsoft Kinect, and publicly available gesture recognition technologies, the interface allows visitors to navigate in three dimensions without introducing any kind of physical device like gloves or 3D pointers. In the paper, we describe the system, the tracking technology and the user interface through the case study of Rediscovering Vrouw Maria project. Rediscovering Vrouw Maria is a gesture-based, interactive, virtual reality simulation that enables visitors to explore an underwater cultural heritage site with an intuitive interface based on simple human gestures and actions. As a multi-disciplinary design research project, it has been undertaken by Department of Media in Aalto University School of Arts, Design and Architecture and the Maritime Archeology Unit in the National Board of Antiquities of Finland. Started in 2010, the project resulted in an installation for an upcoming exhibition at the Maritime Museum of Finland in Kotka in April 2012.

[1]  Thomas B. Moeslund,et al.  A Natural Interface to a Virtual Environment through Computer Vision-Estimated Pointing Gestures , 2001, Gesture Workshop.

[2]  Timo Jokela,et al.  SVSb: simple and visual storyboards: developing a visualisation method for depicting user scenarios , 2007, OZCHI '07.

[3]  Doug A. Bowman,et al.  An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments , 1997, SI3D.

[4]  Slavko Milekic,et al.  Towards Tangible Virtualities: Tangialities. , 2002 .

[5]  R. Hietala,et al.  The Natural Environment of the Shipwreck Vrouw Maria (1771) in the Northern Baltic Sea: an assessment of her state of preservation , 2011 .

[6]  Saul Greenberg,et al.  Prototyping an intelligent agent through Wizard of Oz , 1993, INTERCHI.

[7]  Mark R. Mine,et al.  Virtual Environment Interaction Techniques , 1995 .

[8]  Marc Rettig,et al.  Prototyping for tiny fingers , 1994, CACM.

[9]  Jakob Nielsen,et al.  Iterative user-interface design , 1993, Computer.

[10]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[11]  Thomas B. Moeslund,et al.  A Procedure for Developing Intuitive and Ergonomic Gesture Interfaces for HCI , 2003, Gesture Workshop.

[12]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[13]  Trevor Darrell,et al.  Untethered gesture acquisition and recognition for virtual world manipulation , 2005, Virtual Reality.

[14]  J. Cassell.  Computer Vision for Human–Machine Interaction: A Framework for Gesture Generation and Interpretation , 1998 .

[15]  Norbert A. Streitz,et al.  Roomware: Toward the Next Generation of Human- Computer Interaction Based on an Integrated Design of Real and Virtual Worlds , 2001 .

[16]  A. Pentland,et al.  Computer Vision for Human–Machine Interaction: A Framework for Gesture Generation and Interpretation , 1998 .

[17]  Gora Chand Nandi,et al.  Clustering Method Evaluation for Hidden Markov Model Based Real-Time Gesture Recognition , 2009, 2009 International Conference on Advances in Recent Technologies in Communication and Computing.

[18]  Thomas S. Huang,et al.  Gesture modeling and recognition using finite state machines , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[19]  Carolyn Snyder,et al.  Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces , 2003 .

[20]  J. F. Kelley,et al.  An iterative design methodology for user-friendly natural language office information applications , 1984, TOIS.

[21]  William T. Freeman,et al.  Television control by hand gestures , 1994 .

[22]  Antti Oulasvirta,et al.  Understanding contexts by being there: case studies in bodystorming , 2003, Personal and Ubiquitous Computing.

[23]  Marcelo Knörich Zuffo,et al.  On the usability of gesture interfaces in virtual reality environments , 2005, CLIHC '05.

[24]  Tony Rodrigues.  Theorizing Digital Cultural Heritage: A Critical Discourse , 2008 .

[25]  S. Joy Mountford,et al.  The Art of Human-Computer Interface Design , 1990 .