Intelligent Augmented Reality System based on Speech Recognition

In general, most of the current augmented reality systems can combine 3D virtual scenes with live reality, and users usually interact with 3D objects of the augmented reality (AR) system through image recognition. Although the image-recognition technology has matured enough to allow users to interact with the system, the interaction process is usually limited by the number of patterns used to identify the image. It is not convenient to handle. To provide a more flexible interactive manipulation mode, this study imports the speech-recognition mechanism that allows users to operate 3D objects in an AR system simply by speech. In terms of implementation, the program uses Unity3D as the main development environment and the AR e-Desk as the main development platform. The AR eDesk interacts through the identification mechanism of the reacTIVision and its markers. We use Unity3D to build the required 3D virtual scenes and objects in the AR e-Desk and import the Google Cloud Speech suite to the AR e-Desk system to develop the speech-interaction mechanism. Then, the intelligent AR system is developed. Keywords—Intelligent Augmented Reality, Artificial Intelligence, Image Recognition, Speech Recognition,

[1]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[2]  Lefteris Moussiades,et al.  Α Benchmarking of IBM, Google and Wit Automatic Speech Recognition Systems , 2020, AIAI.

[3]  Elise Lemle,et al.  Virtual Reality and the Future of Entertainment , 2015 .

[4]  Chia-Wei Chu,et al.  Interaction Design in Virtual Reality Game Using Arduino Sensors , 2018 .

[5]  Bogdan Iancu,et al.  Evaluating Google Speech-to-Text API's Performance for Romanian e-Learning Resources , 2019, Informatica Economica.

[6]  Tiago M. Fernández-Caramés,et al.  A Fog Computing and Cloudlet Based Augmented Reality System for the Industry 4.0 Shipyard , 2018, Sensors.

[7]  William Hurst,et al.  Augmented Reality for Enhancing Life Science Education , 2019 .

[9]  Yuki Aoki,et al.  Development of a Teaching Aid for Teaching Dynamic Motion Using the Tango Platform , 2019 .

[10]  Janne Paavilainen,et al.  The Pokémon GO Experience: A Location-Based Augmented Reality Mobile Game Goes Mainstream , 2017, CHI.

[11]  Brian Roark,et al.  Bringing contextual information to google speech recognition , 2015, INTERSPEECH.

[12]  Philippe Dondon,et al.  Design of a low cost LIDAR scanning system for didactical applications , 2019 .

[13]  Andreas M. Kunz,et al.  Postural stability analysis in virtual reality using the HTC vive , 2016, VRST.

[14]  李幼升,et al.  Ph , 1989 .

[15]  Raising the Learning Effects for Learners with Low Entrance Scores using Project-Based Learning in Virtual Reality Practice , 2020 .

[17]  Georgios P. Kladis,et al.  A Greek Voice Recognition Interface for ROV Applications, Using Machine Learning Technologies and the CMU Sphinx Platform , 2018 .

[18]  Abdulmotaleb El-Saddik,et al.  Technical Evaluation of HoloLens for Multimedia: A First Look , 2018, IEEE MultiMedia.

[19]  John B. McGhee,et al.  Journey to the centre of the cell (JTCC): a 3D VR experience derived from migratory breast cancer cell image data , 2016, SIGGRAPH ASIA VR Showcase.