An Augmented Reality Approach to 3D Solid Modeling and Demonstration

This paper presents an intuitive and natural gesture-based methodology for solid modelling in the Augmented Reality (AR) environment. The framework of Client/Server (C/S) is adopted to design the AR-based computer aided design (CAD) system. The method of creating random or constraints- based points using gesture recognition is developed to support modelling. In addition, a prototype system of product 3D solid modelling has been successfully developed, we have compared it with traditional CAD systems through several basic design modeling tasks. Finally, analysis of questionnaire feedback survey shows the intuitiveness and effectiveness of the system, and user studies demonstrate the advantage of helping accomplish the product early design and creating and manipulating the 3D model in the AR environment.

[1]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  Tae-Kyun Kim,et al.  Latent Regression Forest: Structured Estimation of 3D Articulated Hand Posture , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Gilberto Osorio-Gómez,et al.  AIR-MODELLING: A tool for gesture-based solid modelling in context during early design stages in AR environments , 2015, Comput. Ind..

[4]  Sylvain Paris,et al.  6D hands: markerless hand-tracking for computer aided design , 2011, UIST.

[5]  Li Cheng,et al.  Estimate Hand Poses Efficiently from Single Depth Images , 2015, International Journal of Computer Vision.

[6]  Yi-Hsuan Wang,et al.  Exploring the effectiveness of integrating augmented reality-based materials to support writing activities , 2017, Comput. Educ..

[7]  Peter Brandl,et al.  Shared Design Space: Sketching Ideas Using Digital Pens and a Large Augmented Tabletop Setup , 2006, ICAT.

[8]  Jihad El-Sana,et al.  In-place 3D sketching for authoring and augmenting mechanical systems , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[9]  Ting-Chia Hsu,et al.  Learning English with Augmented Reality: Do learning styles matter? , 2017, Comput. Educ..

[10]  Steven K. Feiner,et al.  Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair , 2011, IEEE Transactions on Visualization and Computer Graphics.

[11]  Sang-Heon Lee,et al.  Enhanced hand part classification from a single depth image using random decision forests , 2016, IET Comput. Vis..

[12]  Sen Yang,et al.  A LINE-MOD-based markerless tracking approachfor AR applications , 2017 .

[13]  Andrew Y. C. Nee,et al.  Augmented reality applications in design and manufacturing , 2012 .

[14]  Andrew Y. C. Nee,et al.  A comprehensive survey of augmented reality assembly research , 2016, Advances in Manufacturing.

[15]  Andrew Y. C. Nee,et al.  Augmented reality for collaborative product design and development , 2010 .

[16]  Soh-Khim Ong,et al.  Product information visualization and augmentation in collaborative design , 2008, Comput. Aided Des..

[17]  Soh-Khim Ong,et al.  A mixed reality environment for collaborative product design and development , 2009 .

[18]  Ehud Sharlin,et al.  Napkin sketch: handheld mixed reality 3D sketching , 2008, VRST '08.

[19]  Woontack Woo,et al.  Metaphoric Hand Gestures for Orientation-Aware VR Object Manipulation With an Egocentric Viewpoint , 2017, IEEE Transactions on Human-Machine Systems.

[20]  Ashok K. Goel,et al.  Cognitive, collaborative, conceptual and creative - Four characteristics of the next generation of knowledge-based CAD systems: A study in biologically inspired design , 2012, Comput. Aided Des..

[21]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[22]  Wojciech Moczulski,et al.  Augmented Reality for Machinery Systems Design and Development , 2010, ISPE CE.

[23]  Soh-Khim Ong,et al.  A Framework for Multiple-View Product Representation Using Augmented Reality , 2006, 2006 International Conference on Cyberworlds.

[24]  Mark Billinghurst,et al.  Medical learning murmurs simulation with mobile audible augmented reality , 2016, SIGGRAPH ASIA Mobile Graphics and Interactive Applications.

[25]  Lu Yang,et al.  Survey on 3D Hand Gesture Recognition , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Bruce H. Thomas,et al.  Tinmith-Metro: new outdoor techniques for creating city models with an augmented reality wearable computer , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[27]  Zeyu Li,et al.  Coronary Heart Disease Preoperative Gesture Interactive Diagnostic System Based on Augmented Reality , 2017, Journal of Medical Systems.

[28]  David K. Wright,et al.  Three-dimensional immersive mixed-reality interface for structural design , 2012 .

[29]  Qunsheng Peng,et al.  An approach to solid modeling in a semi-immersive virtual environment , 2000, Comput. Graph..

[30]  Henry Been-Lirn Duh,et al.  Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[31]  Seungwoo Kang,et al.  A System to Analyze Group Socializing Behaviors in Social Parties , 2017, IEEE Transactions on Human-Machine Systems.

[32]  Soh-Khim Ong,et al.  Vision-Based Hand Interaction in Augmented Reality Environment , 2011, Int. J. Hum. Comput. Interact..

[33]  Phillip S. Dunston,et al.  Virtual Visualization for the Mechanical Trade , 2000 .

[34]  Francisco Simões,et al.  Markerless tracking system for augmented reality in the automotive industry , 2017, Expert Syst. Appl..

[35]  Soh-Khim Ong,et al.  Collaborative design in 3D space , 2008, VRCAI '08.

[36]  Li-Wei Chan,et al.  Real-Time 3D Model-Based Gesture Tracking for Multimedia Control , 2010, 2010 20th International Conference on Pattern Recognition.

[37]  Mary Lou Maher,et al.  The Impact of Tangible User Interfaces on Designers' Spatial Cognition , 2008, Hum. Comput. Interact..