EG 2005 Tutorial on Mixed Realities in Inhabited Worlds

We will first explain what is Mixed Reality covering the spectrum from Reality to Virtual Reality. We will emphasize Augmented Reality which augments the user's view of the real world by composing 3D virtual objects with their real world counterparts, necessitating that the user maintains a sense of presence in that world. After a State-ofthe-Art of the main technologies and concepts, we will survey a few applications in medicine, maintenance, tourism, and cultural heritage .

[1]  James C. Lester,et al.  Increasing believability in animated pedagogical agents , 1997, AGENTS '97.

[2]  Bruce Blumberg,et al.  No Bad Dogs: Ethological Lessons for Learning in Hamsterdam* , 1999 .

[3]  J. Averill.  A CONSTRUCTIVIST VIEW OF EMOTION , 1980 .

[4]  Nadia Magnenat-Thalmann,et al.  Personalised real-time idle motion synthesis , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..

[5]  D. Thalmann,et al.  VR haptic interfaces for teleoperation: an evaluation study , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[6]  Hugues Bersini.  Reinforcement learning for homeostatic endogenous variables , 1994 .

[7]  Hans P. Moravec.  Robot Rover Visual Navigation , 1981 .

[8]  Daniel Thalmann,et al.  Synthetic Vision and Audition for Digital Actors , 1995, Comput. Graph. Forum.

[9]  Demetri Terzopoulos,et al.  Physically-based facial modelling, analysis, and animation , 1990, Comput. Animat. Virtual Worlds.

[10]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[11]  Maurizio Mancini,et al.  Formational parameters and adaptive prototype instantiation for MPEG-4 compliant gesture synthesis , 2002, Proceedings of Computer Animation 2002 (CA 2002).

[12]  Marc Cavazza,et al.  Interacting with Virtual Agents in Mixed Reality Interactive Storytelling , 2003, IVA.

[13]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[15]  ThalmannDaniel,et al.  Mixing virtual and real scenes in the site of ancient Pompeii , 2005 .

[16]  Rosalind W. Picard.  Affective Computing , 1997 .

[17]  R. Plutchik.  Emotion, a psychoevolutionary synthesis , 1980 .

[18]  Michael Harrington,et al.  Constellation: a wide-range wireless motion-tracking system for augmented reality and virtual set applications , 1998, SIGGRAPH.

[19]  Jean-Claude Latombe,et al.  Fast synthetic vision, memory, and learning models for virtual humans , 1999, Proceedings Computer Animation 1999.

[20]  Daniel Thalmann,et al.  A Multi-sensor Approach for Grasping and 3D Interaction , 1995, Computer Graphics.

[21]  Nobuyuki Kita,et al.  3D simultaneous localisation and map-building using active vision for a robot moving on undulating terrain , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[22]  Daniel Thalmann,et al.  Evaluation of Gaze Tracking Technology for Social Interaction in Virtual Environments , 2004 .

[23]  Khoi Nguyen,et al.  Computer-vision-based registration techniques for augmented reality , 1996, Other Conferences.

[24]  David W. Murray,et al.  Head pose estimation for wearable robot control , 2002, BMVC.

[25]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[26]  P. Young,et al.  Emotion and personality , 1963 .

[27]  Rachid Deriche,et al.  A computational approach for corner and vertex detection , 1993, International Journal of Computer Vision.

[28]  Cordelia Schmid,et al.  Local Grayvalue Invariants for Image Retrieval , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Marc Alexa,et al.  Linear combination of transformations , 2002, ACM Trans. Graph..

[30]  Daniel Thalmann,et al.  An artificial life environment for autonomous virtual agents with multi‐sensorial and multi‐perceptive features , 2004, Comput. Animat. Virtual Worlds.

[31]  Daniel Thalmann,et al.  Navigation for digital actors based on synthetic vision, memory, and learning , 1995, Comput. Graph..

[32]  M. Slater,et al.  An experiment on fear of public speaking in virtual reality. , 2001, Studies in Health Technology and Informatics.

[33]  Henning Schulzrinne,et al.  RTP: A Transport Protocol for Real-Time Applications , 1996, RFC.

[34]  David E. Breen,et al.  Real-time vision-based camera tracking for augmented reality applications , 1997, VRST '97.

[35]  William J. Cavanaugh,et al.  Roofed theaters of classical antiquity , 1996 .

[36]  Frank Chongwoo Park,et al.  Smooth invariant interpolation of rotations , 1997, TOGS.

[37]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[38]  D. Long.  I of the Vortex: From Neurons to Self , 2002 .

[39]  Patrick Bouthemy,et al.  A 2D-3D model-based approach to real-time visual tracking , 2001, Image Vis. Comput..

[40]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Sarah Hampson,et al.  State of the Art: Personality , 2006 .

[42]  Vincent Lepetit,et al.  Combining edge and texture information for real-time accurate 3D camera tracking , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[43]  Roberto Cipolla,et al.  Real-Time Visual Tracking of Complex Structures , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Andrew Zisserman,et al.  Multi-view Matching for Unordered Image Sets, or "How Do I Organize My Holiday Snaps?" , 2002, ECCV.

[45]  Craig W. Reynolds.  An evolved, vision-based behavioral model of coordinated group motion , 1993 .

[46]  J. Leigh,et al.  Scientists in wonderland: A report on visualization applications in the CAVE virtual reality environment , 1993, Proceedings of 1993 IEEE Research Properties in Virtual Reality Symposium.

[47]  Daniel Thalmann,et al.  Improving user comfort in haptic virtual environments through gravity compensation , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[48]  Blandine de Laval,et al.  STARMATE : Using Augmented Reality technology for computer guided maintenance of complex mechanical elements , 2002 .

[49]  George Papagiannakis,et al.  Mixing virtual and real scenes in the site of ancient Pompeii , 2005, Comput. Animat. Virtual Worlds.

[50]  H. Eysenck.  Biological dimensions of personality. , 1990 .

[51]  David G. Lowe,et al.  Local feature view clustering for 3D object recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[52]  John P. Lewis,et al.  Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation , 2000, SIGGRAPH.

[53]  Bernd Kleinjohann,et al.  Mobile AR4ALL , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[54]  David Zeltzer,et al.  Autonomy, Interaction, and Presence , 1992, Presence: Teleoperators & Virtual Environments.

[55]  Luc Van Gool,et al.  Wide Baseline Stereo Matching based on Local, Affinely Invariant Regions , 2000, BMVC.

[56]  W. Mischel,et al.  A cognitive-affective system theory of personality: reconceptualizing situations, dispositions, dynamics, and invariance in personality structure. , 1995, Psychological review.

[57]  George Papagiannakis,et al.  VHD++ development framework: towards extendible, component based VR/AR simulation engine featuring advanced virtual character technologies , 2003, Proceedings Computer Graphics International 2003.

[58]  Daniel Thalmann,et al.  Joint-dependent local deformations for hand animation and object grasping , 1989 .

[59]  Y Shoda,et al.  Reconciling processing dynamics and personality dispositions. , 1998, Annual review of psychology.

[60]  Zoran Popovic,et al.  Articulated body deformation from range scan data , 2002, SIGGRAPH.

[61]  P. Ekman.  Emotion in the human face , 1982 .

[62]  Wolfgang Friedrich,et al.  ARVIKA-augmented reality for development, production and service , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[63]  E. Gordon,et al.  Social phobics do not see eye to eye: a visual scanpath study of emotional expression processing. , 2003, Journal of anxiety disorders.

[64]  Dinesh Manocha,et al.  V-COLLIDE: accelerated collision detection for VRML , 1997, VRML '97.

[65]  Demetri Terzopoulos,et al.  Perceptual modeling for the behavioral animation of fishes , 1994 .

[66]  Adrian David Cheok,et al.  Online 6 DOF augmented reality registration from natural features , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[67]  J. Gallier,et al.  COMPUTING EXPONENTIALS OF SKEW-SYMMETRIC MATRICES AND LOGARITHMS OF ORTHOGONAL MATRICES , 2002 .

[68]  Andrew W. Fitzgibbon,et al.  Reliable Fiducial Detection in Natural Scenes , 2004, ECCV.

[69]  Georgios Papaioannou,et al.  A Fast Depth-Buffer-Based Voxelization Algorithm , 1999, J. Graphics, GPU, & Game Tools.

[70]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[71]  Hideyuki Tamura,et al.  Mixed Reality: Future Dreams Seen at the Border between Real and Virtual Worlds , 2001, IEEE Computer Graphics and Applications.

[72]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[73]  Daniel Thalmann,et al.  VHD: a system for directing real-time virtual actors , 1999, The Visual Computer.

[74]  Yali Amit,et al.  Shape Quantization and Recognition with Randomized Trees , 1997, Neural Computation.

[75]  Daniel Thalmann,et al.  SMILE: A Multilayered Facial Animation System , 1991, Modeling in Computer Graphics.

[76]  Ioannis T. Christou,et al.  Design and development issues for ARCHEOGUIDE: An Augmented Reality based cultural heritage on-site guide , 2001 .

[77]  Norman I. Badler,et al.  The EMOTE model for effort and shape , 2000, SIGGRAPH.

[78]  Abderrahim Benslimane,et al.  Real-time multimedia services over Internet , 2000, 1st European Conference on Universal Multiservice Networks. ECUMN'2000 (Cat. No.00EX423).

[79]  Angelo Arleo,et al.  Combining Multimodal Sensory Input for Spatial Learning , 2002, ICANN.

[80]  Stefan Schlechtweg,et al.  Non-photorealistic computer graphics: modeling, rendering, and animation , 2002 .

[81]  Daniel Thalmann,et al.  A vision-based approach to behavioural animation , 1990, Comput. Animat. Virtual Worlds.

[82]  Peter-Pike J. Sloan,et al.  Shape by example , 2001, I3D '01.

[83]  Luc Van Gool,et al.  Recognizing color patterns irrespective of viewpoint and illumination , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[84]  B. S. Carlson,et al.  Comparison of modern CCD and CMOS image sensor technologies and systems for low resolution imaging , 2002, Proceedings of IEEE Sensors.

[85]  David G. Lowe,et al.  Shape indexing using approximate nearest-neighbour search in high-dimensional spaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[86]  Alexandre Pouget,et al.  A computational perspective on the neural basis of multisensory spatial representations , 2002, Nature Reviews Neuroscience.

[87]  Stefan Kopp,et al.  Synthesizing multimodal utterances for conversational agents , 2004, Comput. Animat. Virtual Worlds.

[88]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[89]  N. Bobick.  Rotating objects using quaternions , 1998 .

[90]  Max M North,et al.  Virtual Reality Therapy: case study of fear of public speaking. , 2002, Studies in health technology and informatics.

[91]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[92]  Bruce H. Thomas,et al.  ARQuake: an outdoor/indoor augmented reality first person application , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[93]  H. Schlosberg.  A scale for the judgement of facial expressions , 1941 .

[94]  Keith Waters,et al.  A muscle model for animation three-dimensional facial expression , 1987, SIGGRAPH.

[95]  Michael Gleicher,et al.  Building efficient, accurate character skins from examples , 2003, ACM Trans. Graph..

[96]  PRINCETON UNIVERSITY PRESS , 2005 .

[97]  G. A. Thomas,et al.  A versatile camera position measurement system for virtual reality TV production , 1997 .