Immersive modeling system (IMMS) for personal electronic products using a multi-modal interface

In developing new personal electronic products, the development time has shortened to a few months due to high competition in the market. Now it has become very hard to meet the time to market by evaluating products by making physical prototypes. To overcome this problem, we propose an immersive modeling system (IMMS) that enables us to interact with a digital product model in the augmented virtual environment using a multi-modal interface. The IMMS allows the user to evaluate the product model realistically using visual, auditory, and tactile/force senses. The architecture and main modules of the system are described in detail. The integration problems encountered during the development of the test bed are discussed. Application examples to personal electronic products are also included.

[1]  S. H. Choi,et al.  A virtual prototyping system for rapid product development , 2004, Comput. Aided Des..

[2]  Davide Rocchesso,et al.  The Sounding Object , 2002 .

[3]  James A. Larson,et al.  Guidelines for multimodal user interface design , 2004, CACM.

[4]  Leif Kobbelt,et al.  OpenMesh: A Generic and Efficient Polygon Mesh Data Structure , 2002 .

[5]  Oliver Stefani,et al.  Extending the desktop workplace by a portable virtual reality system , 2006, Int. J. Hum. Comput. Stud..

[6]  Woohun Lee,et al.  Augmented foam: a tangible augmented reality for product design , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[7]  James E. Rumbaugh,et al.  State trees as structured finite state machines for user interfaces , 1988, UIST '88.

[8]  Davide Rocchesso,et al.  Controlling Material Properties in Physical Models of Sounding Objects , 2001, ICMC.

[9]  Raffaele de Amicis,et al.  Spacedesign: A Mixed Reality Workspace for Aesthetic Industrial Design , 2002, ISMAR.

[10]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[11]  Huining Feng.  DCHARTS, A FORMALISM FOR MODELING AND SIMULATION BASED DESIGN OF REACTIVE SOFTWARE SYSTEMS , 2004 .

[12]  Tom Page,et al.  An investigation into the implementation of virtual reality technologies in support of conceptual design , 2006 .

[13]  Philip N. Day,et al.  System design and user evaluation of Co-Star: An immersive stereoscopic system for cable harness design , 2007, Comput. Aided Des..

[14]  Karol Myszkowski,et al.  Reverse engineering approach to appearance-based design of metallic and pearlescent paints , 2004, The Visual Computer.

[15]  Thomas Huining Feng.  Modeling and Simulation Based Design with DCharts , .

[16]  Oussama Khatib,et al.  The haptic display of complex graphical environments , 1997, SIGGRAPH.

[17]  Raffaele de Amicis,et al.  Spacedesign: a mixed reality workspace for aesthetic industrial design , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[18]  John Kenneth Salisbury,et al.  Haptic Rendering: Introductory Concepts , 2004, IEEE Computer Graphics and Applications.

[19]  Godfried T. Toussaint,et al.  An Optimal Algorithm for Determining the Visibility of a Polygon from an Edge , 1981, IEEE Transactions on Computers.

[20]  Davide Rocchesso,et al.  MODELING COLLISION SOUNDS: NON-LINEAR CONTACT FORCE , 2001 .

[21]  Wojciech Matusik,et al.  A data-driven reflectance model , 2003, ACM Trans. Graph..

[22]  Martin Bauer,et al.  Fata Morgana - a presentation system for product design , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[23]  Pere Brunet,et al.  MiniVR: a portable virtual reality system , 2004, Comput. Graph..

[24]  Nadine Sarter,et al.  Multimodal information presentation: Design guidance and research challenges , 2006 .

[25]  Ying Zhang,et al.  Evaluation of Multi-sensory Feedback on the Usability of a Virtual Assembly Environment , 2007, J. Multim..

[26]  Jont B. Allen,et al.  Image method for efficiently simulating small‐room acoustics , 1976 .

[27]  Hyungjun Park,et al.  B-spline curve fitting based on adaptive curve refinement using dominant points , 2007, Comput. Aided Des..

[28]  Wendy A Winner.  Introduction to Kernel Density Estimation. , 1985 .

[29]  Richard O. Duda,et al.  An efficient HRTF model for 3-D sound , 1997, Proceedings of 1997 Workshop on Applications of Signal Processing to Audio and Acoustics.

[30]  David Harel,et al.  Statecharts: A Visual Formalism for Complex Systems , 1987, Sci. Comput. Program..

[31]  Davide Rocchesso,et al.  Physically-based audio rendering of contact , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[32]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[33]  Csaba Antonya,et al.  Virtual reality in product design and robotics , 2011, 2011 2nd International Conference on Cognitive Infocommunications (CogInfoCom).

[34]  Enrico Gaia,et al.  Multimodal astronaut virtual training prototype , 2006, Int. J. Hum. Comput. Stud..

[35]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[36]  B. Kapralos,et al.  Auditory Perception and Spatial Auditory Systems4 , 2002 .

[37]  Rakesh Gupta,et al.  Prototyping and design for assembly analysis using multimodal virtual environments , 1997, Comput. Aided Des..

[38]  J. S Bao,et al.  Immersive virtual product development , 2002 .