X-Vision: An Augmented Vision Tool with Real-Time Sensing Ability in Tagged Environments

We present the concept of X-Vision, an enhanced Augmented Reality (AR)-based visualization tool, with the realtime sensing capability in a tagged environment. We envision that this type of a tool will enhance the user-environment interaction and improve the productivity in factories, smartspaces, home & office environments, maintenance/facility rooms and operation theatres, etc. In this paper, we describe the design of this visualization system built upon combining the object's pose information estimated by the depth camera and the object's ID & physical attributes captured by the RFID tags. We built a physical prototype of the system demonstrating the projection of 3D holograms of the objects encoded with sensed information like water-level and temperature of common office/household objects. The paper also discusses the quality metrics used to compare the pose estimation algorithms for robust reconstruction of the object's 3D data.

[1]  Rahul Bhattacharyya,et al.  Low-Cost, Ubiquitous RFID-Tag-Antenna-Based Sensing , 2010, Proceedings of the IEEE.

[2]  Asaf Tzadok,et al.  Fine-Grained Recognition of Thousands of Object Categories with Single-Example Training , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Dieter Fox,et al.  PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes , 2017, Robotics: Science and Systems.

[5]  Graciela Guerrero,et al.  Virtual Touch FlyStick and PrimBox: Two Case Studies of Mixed Reality for Teaching Geometry , 2015, UCAmI.

[6]  R. Bhattacharyya,et al.  Low-cost, automated inventory control of sharps in operating theaters using passive RFID tag-sensors , 2017, 2017 IEEE International Conference on RFID Technology & Application (RFID-TA).

[7]  Ichiro Sakuma,et al.  Augmented Reality Navigation With Automatic Marker-Free Image Registration Using 3-D Image Overlay for Dental Surgery , 2014, IEEE Transactions on Biomedical Engineering.

[8]  Ko-Chiu Wu,et al.  Transform children's library into a mixed-reality learning environment: Using smartwatch navigation and information visualization interfaces , 2017, 2017 Pacific Neighborhood Consortium Annual Conference and Joint Meetings (PNC).

[9]  Silvia Baldiris,et al.  Augmented Reality Applications for Education: Five Directions for Future Research , 2017, AVR.

[10]  Jonathan Dyssel Stets,et al.  Visualization and labeling of point clouds in virtual reality , 2017, SIGGRAPH ASIA.

[11]  Luca Catarinucci,et al.  A Battery-Assisted Sensor-Enhanced RFID Tag Enabling Heterogeneous Wireless Sensor Networks , 2014, IEEE Sensors Journal.

[12]  Rahul Bhattacharyya,et al.  Towards pervasive soil moisture sensing using RFID tag antenna-based sensors , 2015, 2015 IEEE International Conference on RFID Technology and Applications (RFID-TA).

[13]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  James E. Gentle,et al.  Matrix Transformations and Factorizations , 2017 .

[15]  Namho Chung,et al.  Tourists' intention to visit a destination: The role of augmented reality (AR) application for a heritage site , 2015, Comput. Hum. Behav..

[16]  Paul J. Besl,et al.  Method for registration of 3-D shapes , 1992, Other Conferences.

[17]  A. Dunser,et al.  CityViewAR: A mobile outdoor AR application for city visualization , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality - Arts, Media, and Humanities (ISMAR-AMH).

[18]  Thomas Ussmueller,et al.  Battery-less UHF RFID controlled transistor switch for Internet of Things applications — A feasibility study , 2018, 2018 IEEE Topical Conference on Wireless Sensors and Sensor Networks (WiSNet).

[19]  Jean-François Lalonde,et al.  Real-Time High Resolution 3D Data on the HoloLens , 2016, 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct).

[20]  Paul Coulton,et al.  PAC-LAN: mixed-reality gaming with RFID-enabled mobile phones , 2006, CIE.

[21]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[22]  Meng Shi,et al.  Tangible Play Surface Using Passive RFID Sensor Array , 2018, CHI Extended Abstracts.

[23]  Roc Berenguer,et al.  Full Passive UHF Tag With a Temperature Sensor Suitable for Human Body Temperature Monitoring , 2010, IEEE Transactions on Circuits and Systems II: Express Briefs.

[24]  M. Tentzeris,et al.  RFID Passive Gas Sensor Integrating Carbon Nanotubes , 2011 .

[25]  S. N. R. Kantareddy,et al.  Towards low-cost object tracking: Embedded RFID in golf balls using 3D printed masks , 2017, 2017 IEEE International Conference on RFID (RFID).

[26]  Munindar P. Singh,et al.  Augmented Reality Interfaces , 2013, IEEE Internet Computing.

[27]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[28]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.