CoCo - A Framework for Multicore Visuo-Haptics in Mixed Reality

Mixed Reality applications involve the integration of RGB-D streams with virtual entities potentially extended with force feedback. Increasing complexity of the applications pushes the limits of traditional computing structures, not keeping up with the increased computing power of multicore platform. This paper presents the CoCo framework, a component based, multicore system designed for tackling the challenges of visuo-haptics in mixed reality environment, with structural reconfiguration. Special care has been also given to the management of transformation between reference frames for easing registration, calibration and integration of robotic systems. The framework is described together with a description of two relevant case studies.

[1]  Herman Bruyninckx,et al.  Open robot control software: the OROCOS project , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[2]  Emanuele Ruffaldi,et al.  Third Point of View Augmented Reality for Robot Intentions Visualization , 2016, AVR.

[3]  John Kenneth Salisbury,et al.  Voxel-Based Haptic Rendering Using Implicit Sphere Trees , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[4]  John Kenneth Salisbury,et al.  Haptic Rendering: Introductory Concepts , 2004, IEEE Computer Graphics and Applications.

[5]  Walter F. Bischof,et al.  InTml: A Dataflow Oriented Development System for Virtual Reality Applications , 2008, PRESENCE: Teleoperators and Virtual Environments.

[6]  Carlo Alberto Avizzano,et al.  Augmented reality-aided tele-presence system for robot manipulation in industrial manufacturing , 2015, VRST.

[7]  Carlo Alberto Avizzano,et al.  Encountered haptic Augmented Reality interface for remote examination , 2015, 2015 IEEE Symposium on 3D User Interfaces (3DUI).

[8]  K. Salisbury,et al.  Haptic Rendering of Surfaces Defined by Implicit Functions , 1997, Dynamic Systems and Control.

[9]  Massimo Bergamasco,et al.  Lowering the development time of multimodal interactive application: the real-life experience of the XVR project , 2005, ACE '05.

[10]  Carlo Alberto Avizzano,et al.  Co-located haptic interaction for virtual USG exploration , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[11]  Carlo Alberto Avizzano,et al.  Immersive ROS-integrated framework for robot teleoperation , 2015, 2015 IEEE Symposium on 3D User Interfaces (3DUI).

[12]  Jérémie Allard,et al.  FlowVR: A Middleware for Large Scale Virtual Reality Applications , 2004, Euro-Par.

[13]  Ruffaldi Emanuele,et al.  Co-located haptic interaction for virtual USG exploration , 2015, EMBC 2015.

[14]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[15]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[16]  Ulrich Bockholt,et al.  Instantreality — A Framework for Industrial Augmented and Virtual Reality Applications , 2011 .

[17]  Oussama Khatib,et al.  The haptic display of complex graphical environments , 1997, SIGGRAPH.

[18]  Antonio Frisoli,et al.  A haptic toolkit for the development of immersive and web-enabled games , 2006, VRST '06.

[19]  Carlo Alberto Avizzano,et al.  Encountered-type haptic interface for virtual interaction with real objects based on implicit surface haptic rendering for remote palpation , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[20]  R. Osfield,et al.  Tutorial: Open scene graph A: introduction tutorial: Open scene graph B: examples and applications , 2004, IEEE Virtual Reality 2004.

[21]  Ulrich Eck,et al.  HARP: A framework for visuo-haptic augmented reality , 2013, 2013 IEEE Virtual Reality (VR).

[22]  Mark Billinghurst,et al.  ComposAR: An intuitive tool for authoring AR applications , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.