Principles, interactions and devices for real-world immersive modeling

Abstract Building a real-world immersive 3D modeling application is hard. In spite of the many supposed advantages of working in the virtual world, users quickly tire of waving their arms about and the resulting models remain simplistic at best. The dream of creation at the speed of thought has largely remained unfulfilled due to numerous factors such as the lack of suitable menu and system controls, inability to perform precise manipulations, lack of numeric input, challenges with ergonomics, and difficulties with maintaining user focus and preserving immersion. The focus of our research is on the building of virtual world applications that can go beyond the demo and can be used to do real-world work. The goal is to develop interaction techniques that support the richness and complexity required to build complex 3D models, yet minimize expenditure of user energy and maximize user comfort. We present an approach that combines the natural and intuitive power of virtual reality (VR) interaction, the precision and control of 2D touch surfaces, and the richness of a commercial modeling package. We discuss the benefits of collocating 2D touch with 3D bimanual spatial input, the challenges in designing a custom controller targeted at achieving the same, and the new avenues that this collocation creates. We describe our Touch Where You Can technique that adapts the user interface to support a wide array of hand sizes, minimizing the ergonomic impact on the user. Finally, we demonstrate new interface designs that are better suited for the thumbs-only touch interactions favored by our system.