3D object recognition for Virtual Reality based Digital Twins

Abstract: Recent advances in VR/AR have fueled the interest in automating the development of 3D interfaces. An important step of this automation is 3D object recognition. This allows applications to pre-configure assets based on expected behaviours, rather than requiring human developers to hard code such assets. This paper presents a clustering approach for reliably identifying 3D objects. Model-based descriptors are used to extract key features and geometric data are used to develop models for 3D object recognition. Two clustering approaches are used in this paper: hierarchical and K-means. The models are tested on two industrial gearboxes and an accuracy score is calculated to test model performance. The two gearboxes were too small resulting in the accuracy of both algorithms being identical, however the computational efficiency of the K-means algorithm makes it a more suitable choice. The accuracy for both algorithms was 74% when tested on a combined dataset of both gearboxes. This increased to 81% when difficult to identify parts are removed. The model can reliably identify geometrically consistent parts like ball bearings, roller bearings and screws.

[1]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Richard McClatchey,et al.  Intelligent grid enabled services for neuroimaging analysis , 2013, Neurocomputing.

[3]  Ashiq Anjum,et al.  The Clarens Grid-enabled Web Services Framework : Services and Implementation , 2005 .

[4]  Franca Giannini,et al.  A methodology for part classification with supervised machine learning , 2018, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[5]  Jacek Chmielewski.  Finding interactive 3D objects by their interaction properties , 2012, Multimedia Tools and Applications.

[6]  Omer F. Rana,et al.  RES: Real-Time Video Stream Analytics Using Edge Enhanced Clouds , 2020, IEEE Transactions on Cloud Computing.

[7]  Richard Hill,et al.  Improving Cloud Datacentre Scalability, Agility and Performance Using OpenFlow , 2012, 2012 Fourth International Conference on Intelligent Networking and Collaborative Systems.

[8]  Michele Dassisti,et al.  Digital twin paradigm: A systematic literature review , 2021, Comput. Ind..

[9]  Bernard Chazelle,et al.  Shape distributions , 2002, TOGS.

[10]  Ye Duan,et al.  PointGrid: A Deep Network for 3D Shape Understanding , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Ashiq Anjum,et al.  Grid Enabled Analysis : Architecture, prototype and status , 2005 .

[12]  Jun Wang,et al.  Intelligent data fusion algorithm based on hybrid delay-aware adaptive clustering in wireless sensor networks , 2020, Future Gener. Comput. Syst..

[13]  Liwei Wang,et al.  Learning Relationships for Multi-View 3D Object Recognition , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[14]  Nikos A. Vlassis,et al.  The global k-means clustering algorithm , 2003, Pattern Recognit..

[15]  J. Arora,et al.  The Digital Twin , 2023, ATZ worldwide.

[16]  Yue Gao,et al.  GVCNN: Group-View Convolutional Neural Networks for 3D Shape Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Masaki Aono,et al.  Multi-Fourier spectra descriptor and augmentation with spectral clustering for 3D shape retrieval , 2009, The Visual Computer.