Gamified AR/VR Character Rendering and Animation-Enabling Technologies

In recent years, the popularity of mixed reality (MR) environments has increased as they provide attractive and immersive experiences for educational, entertainment, and training purposes. The increasing advances of Augmented Reality (AR) and Virtual Reality (VR) hardware and software technologies also constitute an interesting area of research. In this work, we present the main pipeline followed for creating virtual character-based AR experiences, specifically in cultural heritage environments. Our main goal in this chapter is to compare different software methodologies for creating VR environments and present a complete novel methodology for authoring life-sized AR virtual characters and life-sized AR crowd simulation using only modern mobile devices. One important aspect of these environments that we focus on is creating realistic and interactive virtual characters via procedurally generated body and facial animations which are illuminated with real environment light. Virtual characters' transformations are handled efficiently using a single mathematical framework, the 3D Euclidean geometric algebra (GA), and the conformal geometric algebra (CGA) which is able to handle translations, rotations, and dilations. Using such a single algebraic framework, we avoid conversions between different mathematical representations; as a result, we achieve more efficient performance. We also compare the efficiency of different GA code generators—(a) the Gaigen library, (b) libvsr, and (c) Gaalop—so that a future user of GA can choose the most appropriate, currently available s/w library that will provide the most optimal and efficient results. Our main research involves the following questions: (a) Are novel, low-cost, modern HMDs suitable as VR platforms? (b) Which are the most appropriate s/w platforms needed to realize such VR digital heritage gamified experiences? (c) Can we achieve more efficient AR scene authoring? (d) Can we achieve more efficient AR animation interpolation and skinning using a single mathematical framework employing GA?

[1]  Jirí Zára,et al.  Geometric skinning with approximate dual quaternion blending , 2008, TOGS.

[2]  Dieter Schmalstieg,et al.  Sketching up the world: in situ authoring for mobile Augmented Reality , 2012, Personal and Ubiquitous Computing.

[3]  Joan Lasenby,et al.  Applications of Conformal Geometric Algebra in Computer Vision and Graphics , 2004, IWMM/GIAE.

[4]  D. Hestenes,et al.  Clifford Algebra to Geometric Calculus , 1984 .

[5]  Pan Hui,et al.  Mobile augmented reality survey: a bottom-up approach , 2013, ArXiv.

[6]  George Papagiannakis,et al.  A fast and robust pipeline for populating mobile AR scenes with gamified virtual characters , 2015, SIGGRAPH Asia Mobile Graphics and Interactive Applications.

[7]  Lakhmi C. Jain,et al.  Technologies of Inclusive Well-Being at the Intersection of Serious Games, Alternative Realities, and Play Therapy , 2014, Technologies of Inclusive Well-Being.

[8]  George Papagiannakis,et al.  An inclusive Conformal Geometric Algebra GPU animation interpolation and deformation algorithm , 2016, The Visual Computer.

[9]  George Papagiannakis,et al.  Presence and interaction in mixed reality environments , 2007, The Visual Computer.

[10]  George Papagiannakis,et al.  A Comparison of Gamified, Immersive VR Curation Methods for Enhanced Presence and Human-computer Interaction in Digital Humanities , 2015 .

[11]  Panos E. Trahanias,et al.  glGA: an OpenGL Geometric Application Framework for a Modern, Shader-based Computer Graphics Curriculum , 2014, Eurographics.

[12]  Panos Trahanias,et al.  Mixed-reality geometric algebra animation methods for gamified intangible heritage , 2014 .

[13]  Ari Shapiro,et al.  Building a Character Animation System , 2011, MIG.

[14]  Dinesh Manocha,et al.  Interactive navigation of multiple agents in crowded environments , 2008, I3D '08.

[15]  K. Kanatani.  Understanding Geometric Algebra: Hamilton, Grassmann, and Clifford for Computer Vision and Graphics , 2015 .

[16]  Antonis A. Argyros,et al.  Tracking the articulated motion of two strongly interacting hands , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  George Papagiannakis,et al.  Modeling of bodies and clothes for virtual environments , 2004, 2004 International Conference on Cyberworlds.

[18]  Bruce H. Thomas,et al.  A survey of visual, mixed, and augmented reality gaming , 2012, CIE.

[19]  George Papagiannakis,et al.  A Conformal Geometric Algebra Code Generator Comparison for Virtual Character Simulation in Mixed Reality , 2017 .

[20]  Dinesh Manocha,et al.  Reciprocal Velocity Obstacles for real-time multi-agent navigation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[21]  Maria Pateraki,et al.  Visual estimation of pointed targets for robot guidance via fusion of face pose and hand orientation , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[22]  Joan Lasenby,et al.  Mesh Vertex Pose and Position Interpolation Using Geometric Algebra , 2008, AMDO.

[23]  Yuyu Xu,et al.  Fast, automatic character animation pipelines , 2014, Comput. Animat. Virtual Worlds.

[24]  Daniel Thalmann,et al.  Joint-dependent local deformations for hand animation and object grasping , 1989 .

[25]  Mark Billinghurst,et al.  A component based framework for mobile outdoor AR applications , 2013, VRCAI '13.

[26]  George Papagiannakis,et al.  Geometric algebra rotors for skinned character animation blending , 2013, SIGGRAPH ASIA Technical Briefs.