A Review of NASA Human-Robot Interaction in Space

This review provides an overview of the motivation, challenges, state-of-the-art, and recent research for human-robot interaction (HRI) in space. For context, we focus on NASA space missions, use cases, and systems (both flight and research). However, the discussion is broadly applicable to all activities in space that require or make use of human-robot teams. To date, HRI in space has largely been limited to remote interaction between humans on Earth and robots in space. This interaction is associated with telerobotic operations—from direct (manual) control to intermittent, supervisory control. Recent work, however, has begun to address a wide range of human-robot arrangements (co-located, remote, 1:1, groups, etc.). In addition, researchers have been studying human-robot teaming theory and system design, efficient interaction methods, and human-robot communication. We begin by describing NASA’s use of robots in space for both deep space science and human exploration. We then describe several aspects of HRI that are important for space missions, with emphasis on factors that are critical or unique for the space environment. Next, we provide a brief overview of HRI associated with space systems, including technology demonstrations. Finally, we conclude with a short survey of recent research, which will affect human-robot interaction for both Artemis missions and future missions to Mars.

[1]  Matthias Scheutz,et al.  Shared Mental Models to Support Distributed Human-Robot Teaming in Space , 2018, 2018 AIAA SPACE and Astronautics Forum and Exposition.

[2]  Charles M. Macal,et al.  Modeling Dynamic Multiscale Social Processes in Agent-Based Models , 2008, IEEE Intelligent Systems.

[3]  Sonia Chernova,et al.  A Comparison of Remote Robot Teleoperation Interfaces for General Object Manipulation , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[4]  Robert O. Ambrose,et al.  Robonaut 2 - The first humanoid robot in space , 2011, 2011 IEEE International Conference on Robotics and Automation.

[5]  Terrence Fong,et al.  Peer-to-Peer Human-Robot Interaction for Space Exploration , 2004, AAAI Technical Report.

[6]  Illah R. Nourbakhsh,et al.  Interaction challenges in human-robot space exploration , 2005, INTR.

[7]  David L. Akin,et al.  Space Telerobotics: Unique Challenges to Human–Robot Collaboration in Space , 2013 .

[8]  Kandyce Goodliff,et al.  The Artemis Program: An Overview of NASA's Activities to Return Humans to the Moon , 2020, 2020 IEEE Aerospace Conference.

[9]  Douglas A. Craig,et al.  Deep space gateway concept: Extending human presence into cislunar space , 2018, 2018 IEEE Aerospace Conference.

[10]  Terrence Fong,et al.  Results from Testing Crew-Controlled Surface Telerobotics on the International Space Station , 2014 .

[11]  Anthony Stentz,et al.  Global planning on the Mars Exploration Rovers: Software integration and surface testing , 2009 .

[12]  Sonia Chernova,et al.  Human-Centric Active Perception for Autonomous Observation , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Julia Badger,et al.  Robonaut 2 on the International Space Station: Status Update and Preparations for IVA Mobility , 2013 .

[14]  D.A. Harrison,et al.  Next Generation Rover for Lunar Exploration , 2008, 2008 IEEE Aerospace Conference.

[15]  Benjamin B. Reed,et al.  NASA’s In-Space Robotic Servicing , 2015 .

[16]  Terrence Fong,et al.  Measuring robot performance in real-time for NASA robotic reconnaissance operations , 2009, PerMIS.

[17]  Terrence Fong,et al.  Flailing, Hailing, Prevailing: Perceptions of Multi-Robot Failure Recovery Strategies , 2021, HRI.

[18]  Jennifer Lee,et al.  Communicating Robot Motion Intent with Augmented Reality , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[19]  Terrence Fong,et al.  Measuring Performance in Real Time during Remote Human-Robot Operations with Adjustable Autonomy , 2010, IEEE Intelligent Systems.

[20]  Trey Smith,et al.  Astrobee: A New Platform for Free-Flying Robotics on the International Space Station , 2016 .

[21]  Sonia Chernova,et al.  A Tale of Two Suggestions: Action and Diagnosis Recommendations for Responding to Robot Failure , 2020, 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN).

[22]  R. Anderson,et al.  Mars Science Laboratory Mission and Science Investigation , 2012 .

[23]  Mark Maimone,et al.  Driving Curiosity: Mars Rover Mobility Trends During the First Seven Years , 2020, 2020 IEEE Aerospace Conference.

[24]  Terrence Fong,et al.  Effects of Robot Sound on Auditory Localization in Human-Robot Collaboration , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[25]  Terrence Fong,et al.  Toward Genuine Robot Teammates: Improving Human-Robot Team Performance Using Robot Shared Mental Models , 2020, AAMAS.

[26]  Terrence Fong,et al.  Design Issues for Real-Time Remote Robotic Science Operations Support Tools: Observations from the Field , 2016 .

[27]  Karen M. Feigh,et al.  Computational Methodology for the Allocation of Work and Interaction in Human-Robot Teams , 2019, Journal of Cognitive Engineering and Decision Making.

[28]  Terrence Fong,et al.  A Survey of Methods for Safe Human-Robot Interaction , 2017, Found. Trends Robotics.

[29]  Michael A. Goodrich,et al.  Toward Human-Robot Interface Standards II: An Examination of Common Elements in Human-Robot Interactions Across the Space Enterprise , 2006 .

[30]  Mark W. Powell,et al.  Mars Exploration Rover Operations with the Science Activity Planner , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[31]  Hooman Hedayati,et al.  Improving Collocated Robot Teleoperation with Augmented Reality , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[32]  William Baker,et al.  Supervisory control of a humanoid robot in microgravity for manipulation tasks , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[33]  Maja J. Matarić,et al.  Generating Expressive Light Signals for Appearance-Constrained Robots , 2018, ISER.

[34]  Illah R. Nourbakhsh,et al.  A Preliminary Study of Peer-to-Peer Human-Robot Interaction , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[35]  Bilge Mutlu,et al.  Designing planning and control interfaces to support user collaboration with flying robots , 2017, Int. J. Robotics Res..

[36]  Stefanie Tellex,et al.  Grounding natural language instructions to semantic goal representations for abstraction and generalization , 2018, Auton. Robots.

[37]  Michael A. Goodrich,et al.  Toward Human-Robot Interface Standards: Use of Standardization and Intelligent Subsystems for Advancing Human-Robotic Competency in Space Exploration , 2006 .

[38]  Phillip Callen Robotic Transfer and Interfaces for External ISS Payloads , 2014 .

[39]  Anthony Stentz,et al.  Global planning on the Mars Exploration Rovers: Software integration and surface testing , 2009, J. Field Robotics.

[40]  Bilge Mutlu,et al.  Communicating Directionality in Flying Robots , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[41]  Terrence Fong,et al.  Smart SPHERES: A Telerobotic Free-Flyer for Intravehicular Activities in Space , 2013 .

[42]  Simon J. Julier,et al.  Everybody Needs Somebody Sometimes: Validation of Adaptive Recovery in Robotic Space Operations , 2019, IEEE Robotics and Automation Letters.

[43]  Michael McCurdy Planning tools for Mars surface operations: Human-Computer Interaction lessons learned , 2009, 2009 IEEE Aerospace conference.

[44]  Sonia Chernova,et al.  Leveraging depth data in remote robot teleoperation interfaces for general object manipulation , 2019, Int. J. Robotics Res..

[45]  Tod Milam,et al.  Techniques and Tools for Summarizing Performance of Robots Operating Remotely , 2016 .

[46]  Amin Maghareh,et al.  Resilience-oriented Design of Extraterrestrial Habitat Systems , 2019, AIAA Propulsion and Energy 2019 Forum.

[47]  Jessica J. Marquez,et al.  Human-Automation Allocations for Current Robotic Space Operations , 2018 .

[48]  Maja J. Matarić,et al.  A Survey of Nonverbal Signaling Methods for Non-Humanoid Robots , 2018, Found. Trends Robotics.

[49]  R. O. Ambrose,et al.  Robonaut 2 — Initial activities on-board the ISS , 2012, 2012 IEEE Aerospace Conference.

[50]  Steven E. Fredrickson,et al.  Application of the mini AERCam free flyer for orbital inspection , 2004, SPIE Defense + Commercial Sensing.