Detection of Nuclear Sources by UAV Teleoperation Using a Visuo-Haptic Augmented Reality Interface

A visuo-haptic augmented reality (VHAR) interface is presented enabling an operator to teleoperate an unmanned aerial vehicle (UAV) equipped with a custom CdZnTe-based spectroscopic gamma-ray detector in outdoor environments. The task is to localize nuclear radiation sources, whose location is unknown to the user, without the close exposure of the operator. The developed detector also enables identification of the localized nuclear sources. The aim of the VHAR interface is to increase the situation awareness of the operator. The user teleoperates the UAV using a 3DOF haptic device that provides an attractive force feedback around the location of the most intense detected radiation source. Moreover, a fixed camera on the ground observes the environment where the UAV is flying. A 3D augmented reality scene is displayed on a computer screen accessible to the operator. Multiple types of graphical overlays are shown, including sensor data acquired by the nuclear radiation detector, a virtual cursor that tracks the UAV and geographical information, such as buildings. Experiments performed in a real environment are reported using an intense nuclear source.

[1]  Robert E. Mahony,et al.  Representation of vehicle dynamics in haptic teleoperation of aerial robots , 2013, 2013 IEEE International Conference on Robotics and Automation.

[2]  L. Marchini,et al.  Study of Surface Treatment Effects on the Metal-CdZnTe Interface , 2009, IEEE Transactions on Nuclear Science.

[3]  Eric W. Frew,et al.  Networking Issues for Small Unmanned Aircraft Systems , 2009, J. Intell. Robotic Syst..

[4]  Tarek Hamel,et al.  Bilateral haptic teleoperation of VTOL UAVs , 2013, 2013 IEEE International Conference on Robotics and Automation.

[5]  Rogelio Lozano,et al.  Outdoor haptic teleoperation of a hexarotor UAV , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[6]  S Kuukankorpi,et al.  Design of a radiation surveillance unit for an unmanned aerial vehicle. , 2005, Journal of environmental radioactivity.

[7]  Huang Wei,et al.  Interactive registration for Augmented Reality GIS , 2012, 2012 International Conference on Computer Vision in Remote Sensing.

[8]  Stefanie Zollmann,et al.  FlyAR: Augmented Reality Supported Micro Aerial Vehicle Navigation , 2014, IEEE Transactions on Visualization and Computer Graphics.

[9]  Stefano Caselli,et al.  Evaluation of a haptic interface for UAV teleoperation in detection of radiation sources , 2016, 2016 18th Mediterranean Electrotechnical Conference (MELECON).

[10]  Antonio Franchi,et al.  An evaluation of haptic cues on the tele-operator's perceptual awareness of multiple UAVs' environments , 2011, 2011 IEEE World Haptics Conference.

[11]  K. Boudergui,et al.  Development of a drone equipped with optimized sensors for nuclear and radiological risk characterization , 2011, 2011 2nd International Conference on Advancements in Nuclear Instrumentation, Measurement Methods and their Applications.

[12]  Yukihisa Sanada,et al.  Aerial radiation monitoring around the Fukushima Dai-ichi Nuclear Power Plant using an unmanned helicopter. , 2015, Journal of environmental radioactivity.

[13]  Stefano Caselli,et al.  Haptic guided UAV for detection of radiation sources in outdoor environments , 2015, 2015 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS).

[14]  John S. Fardoulis,et al.  The use of unmanned aerial systems for the mapping of legacy uranium mines. , 2015, Journal of environmental radioactivity.

[15]  Stefano Stramigioli,et al.  On Bilateral Teleoperation of Aerial Robots , 2014, IEEE Transactions on Robotics.

[16]  Peter I. Corke,et al.  A novel approach to haptic tele-operation of aerial robot vehicles , 2010, 2010 IEEE International Conference on Robotics and Automation.

[17]  Yangquan Chen,et al.  Low-cost Multi-UAV Technologies for Contour Mapping of Nuclear Radiation Field , 2013, J. Intell. Robotic Syst..

[18]  Karol Jędrasiak,et al.  Augmented Reality in UAVs Applications , 2016 .

[19]  Kevin Kochersberger,et al.  Radiation Mapping in Post-Disaster Environments Using an Autonomous Helicopter , 2012, Remote. Sens..

[20]  P. Wong,et al.  A UAV-Mounted Whole Cell Biosensor System for Environmental Monitoring Applications , 2015, IEEE Transactions on NanoBioscience.

[21]  Robert E. Mahony,et al.  Comparative Study of Haptic Interfaces for Bilateral Teleoperation of VTOL Aerial Robots , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[23]  Nak Young Chong,et al.  Fast radiation mapping and multiple source localization using topographic contour map and incremental density estimation , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Vincenzo Lippiello,et al.  Robot Vision: Obstacle-Avoidance Techniques for Unmanned Aerial Vehicles , 2013, IEEE Robotics & Automation Magazine.

[25]  Imad H. Elhajj,et al.  Enhanced teleoperation of UAVs with haptic feedback , 2015, 2015 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[26]  I. Kuvvetli,et al.  A 3D CZT high resolution detector for x- and gamma-ray astronomy , 2014, Astronomical Telescopes and Instrumentation.

[27]  Naokazu Yokoya,et al.  Augmented telepresence using autopilot airship and omni-directional camera , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[28]  Horst Bischof,et al.  Augmented Reality for Construction Site Monitoring and Documentation , 2014, Proceedings of the IEEE.

[29]  René van Paassen,et al.  Artificial Force Field for Haptic Feedback in UAV Teleoperation , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[30]  T. Fujii,et al.  Application of a CZT detector to in situ environmental radioactivity measurement in the Fukushima area. , 2015, Radiation protection dosimetry.

[31]  Jochen H. Schiller,et al.  Micro-drone for the characterization and self-optimizing search of hazardous gaseous substance sources: A new approach to determine wind speed and direction , 2010, 2010 IEEE International Workshop on Robotic and Sensors Environments.

[32]  Stefano Stramigioli,et al.  Kinetic scrolling-based position mapping for haptic teleoperation of unmanned aerial vehicles , 2012, 2012 IEEE International Conference on Robotics and Automation.

[33]  Kari Peräjärvi,et al.  Radiation surveillance using an unmanned aerial vehicle. , 2009, Applied radiation and isotopes : including data, instrumentation and methods for use in agriculture, industry and medicine.

[34]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[35]  Luce Morin,et al.  GPS, GIS and Video Registration for Building Reconstruction , 2007, 2007 IEEE International Conference on Image Processing.

[36]  T. Scott,et al.  Lightweight aerial vehicles for monitoring, assessment and mapping of radiation anomalies. , 2014, Journal of environmental radioactivity.

[37]  A. Hossain,et al.  CdZnTe room-temperature semiconductor gamma-ray detector for national-security applications , 2007, 2007 IEEE Long Island Systems, Applications and Technology Conference.

[38]  Antonio Franchi,et al.  A semi-autonomous UAV platform for indoor remote operation with visual and haptic feedback , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[39]  Tatsuo Torii,et al.  A Remote Radiation Monitoring System Using an Autonomous Unmanned Helicopter for Nuclear Emergencies , 2008 .

[40]  D. Richards,et al.  Low altitude unmanned aerial vehicle for characterising remediation effectiveness following the FDNPP accident. , 2016, Journal of environmental radioactivity.

[41]  Peter I. Corke,et al.  Intercontinental haptic teleoperation of a flying vehicle: A step towards real-time applications , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[42]  Ezio Caroli,et al.  Progress in the Development of CdTe and CdZnTe Semiconductor Radiation Detectors for Astrophysical and Medical Applications , 2009, Sensors.

[43]  Antonio Franchi,et al.  Semi-autonomous trajectory generation for mobile robots with integral haptic shared control , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[44]  Ira S. Moskowitz,et al.  Real-time unmanned aerial vehicle 3D environment exploration in a mixed reality environment , 2016, 2016 International Conference on Unmanned Aircraft Systems (ICUAS).

[45]  Steven Zhiying Zhou,et al.  Model-based localization and drift-free user tracking for outdoor augmented reality , 2010, 2010 IEEE International Conference on Multimedia and Expo.