A Gesture-Based Interface for Seamless Communication between Real and Virtual Worlds

This paper proposes a gesture-based direct manipulation interface that can be used for data transfer among informational artifacts. "Grasp and Drop (Throw)" by hand gestures allows a user to grasp an object on a computer screen and drop (throw) it on other artifacts without touching them. Using the interface, a user can operate some artifacts in the mixed reality world in a seamless manner, and learn this interaction style easily. Based on this interaction technique, we developed a prototype of presentation system using Microsoft PowerPoint, a wall size screen, computer screens and a printer. The presentation system with gestures allows a presenter to navigate through PowerPoint slides and transfer a slide from one computer screen to another. We conducted an experiment which evaluate the interaction style of gestures and analyzed the user's satisfaction with a questionnaire. The result shows that the overall mean of successful recognition is 96.9 %, and the learning of the system is easy.