Human-Robot Teaming: Approaches from Joint Action and Dynamical Systems

As robots start to work alongside people, they are expected to coordinate fluently with humans in teams. Many researchers have explored the problems involved in building more interactive and cooperative robots. In this chapter, we discuss recent work and the main application areas in human-robot teaming. We also shed light on some practical challenges to achieving fluent human-robot coordination, and conclude the chapter with future directions for approaching these problems.

[1]  Alois Knoll,et al.  Applications in HHI: Physical Cooperation , 2018, Humanoid Robotics: A Reference.

[2]  Joseph A. Bulbulia,et al.  Synchronized arousal between performers and related spectators in a fire-walking ritual , 2011, Proceedings of the National Academy of Sciences.

[3]  Laurel D. Riek,et al.  Role Distribution in Synchronous Human-Robot Joint Action , 2014 .

[4]  G. Knoblich,et al.  Action coordination in groups and individuals: learning anticipatory control. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[5]  L. Riek,et al.  Real-Time Empathy : Facial Mimicry on a Robot , 2008 .

[6]  Laurel D. Riek,et al.  The Social Co-Robotics Problem Space : Six Key Challenges , 2010 .

[7]  E. Burdet,et al.  A Framework to Describe, Analyze and Generate Interactive Motor Behaviors , 2012, PloS one.

[8]  Siddhartha S. Srinivasa,et al.  Deliberate Delays During Robot-to-Human Handovers Improve Compliance With Gaze Communication , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[9]  Sandra Hirche,et al.  Synchronization in a goal-directed task: Human movement coordination with each other and robotic partners , 2011, 2011 RO-MAN.

[10]  Jake K. Aggarwal,et al.  Robot-Centric Activity Prediction from First-Person Videos: What Will They Do to Me? , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[11]  Julie A. Shah,et al.  Human-robot co-navigation using anticipatory indicators of human walking motion , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Laurel D. Riek,et al.  Coordination Dynamics in Multi-human Multi-robot Teams , 2016 .

[13]  Günther Knoblich,et al.  Joint Action in Humans: A Model for Human-Robot Interactions , 2017 .

[14]  Cynthia Breazeal,et al.  An Empirical Analysis of Team Coordination Behaviors and Action Planning With Application to Human–Robot Teaming , 2010, Hum. Factors.

[15]  N. Sebanz,et al.  Psychological research on joint action: Theory and data , 2011 .

[16]  Gerald Seet,et al.  An Implementation of Seamless Human-Robot Interaction for Telerobotics , 2008 .

[17]  Laurel D. Riek,et al.  Detecting social context: A method for social event classification using naturalistic multimodal data , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[18]  Oliver Kroemer,et al.  Interaction primitives for human-robot cooperation tasks , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Giulio Sandini,et al.  Movement-Based Communication for Humanoid-Human Interaction , 2018, Humanoid Robotics: A Reference.

[20]  Cordula Vesper,et al.  A minimal architecture for joint action , 2010, Neural Networks.

[21]  Peter Robinson,et al.  When my robot smiles at me Enabling human-robot rapport via real-time head gesture mimicry , 2009 .

[22]  S. Hirche,et al.  Rhythm Patterns Interaction - Synchronization Behavior for Human-Robot Joint Action , 2014, PloS one.

[23]  Frank Broz,et al.  Enriching the human-robot interaction loop with natural, semantic and symbolic gestures , 2017 .

[24]  Guy Hoffman,et al.  Interactive improvisation with a robotic marimba player , 2011, Auton. Robots.

[25]  Peter Robinson,et al.  Cooperative gestures: Effective signaling for humanoid robots , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[26]  D. DeSteno,et al.  The rhythm of joint action: Synchrony promotes cooperative ability , 2010 .

[27]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[28]  Scott Niekum,et al.  Incremental Semantically Grounded Learning from Demonstration , 2013, Robotics: Science and Systems.

[29]  Laurel D. Riek,et al.  Enabling Synchronous Joint Action in Human-Robot Teams , 2015, HRI.

[30]  Laurel D. Riek,et al.  Faster robot perception using Salient Depth Partitioning , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[31]  Karon E. MacLean,et al.  Meet Me where I’m Gazing: How Shared Attention Gaze Affects Human-Robot Handover Timing , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[32]  Laurel D. Riek,et al.  A Model for Time-Synchronized Sensing and Motion to Support Human-Robot Fluency , 2014 .

[33]  HoffmanGuy,et al.  Interactive improvisation with a robotic marimba player , 2011 .

[34]  Wei-Yun Yau,et al.  Human Action Recognition With Video Data: Research and Evaluation Challenges , 2014, IEEE Transactions on Human-Machine Systems.

[35]  Guy Hoffman,et al.  Computational Human-Robot Interaction , 2016, Found. Trends Robotics.

[36]  Cynthia Breazeal,et al.  Improved human-robot team performance using Chaski, A human-inspired plan execution system , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[37]  Anthony G. Pipe,et al.  Joint action understanding improves robot-to-human object handover , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  Laurel D. Riek,et al.  Social context perception for mobile robots , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[39]  T. Nomura.  Empathy as Signalling Feedback Between Humanoid Robots and Humans , 2018, Humanoid Robotics: A Reference.

[40]  Ana Paiva,et al.  Modelling empathic behaviour in a robotic game companion for children: An ethnographic study in real-world settings , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[41]  Cory J. Hayes,et al.  Exploring implicit human responses to robot mistakes in a learning from demonstration task , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[42]  Michael J. Richardson,et al.  Measuring group synchrony: a cluster-phase method for analyzing multivariate movement time-series , 2012, Front. Physio..

[43]  Chao Zhai,et al.  Dynamic similarity promotes interpersonal coordination in joint action , 2015, Journal of The Royal Society Interface.

[44]  Natalie Sebanz,et al.  Prediction in Joint Action: What, When, and Where , 2009, Top. Cogn. Sci..

[45]  Laurel D. Riek,et al.  Object Proposal Algorithms in the Wild: Are they Generalizable to Robot Perception? , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[46]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[47]  Siddhartha S. Srinivasa,et al.  Using spatial and temporal contrast for fluent robot-human hand-overs , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[48]  Giacomo Novembre,et al.  Motor simulation and the coordination of self and other in real-time joint action. , 2014, Social cognitive and affective neuroscience.

[49]  H. Bekkering,et al.  Exploring the brain basis of joint action: Co-ordination of actions, goals and intentions , 2007, Social neuroscience.

[50]  Stefanos Nikolaidis,et al.  Efficient Model Learning for Human-Robot Collaborative Tasks , 2014, ArXiv.

[51]  Takayuki Kanda,et al.  A Model of Distributional Handing Interaction for a Mobile Robot , 2013, Robotics: Science and Systems.

[52]  Laurel D. Riek,et al.  Movement Coordination in Human–Robot Teams: A Dynamical Systems Approach , 2016, IEEE Transactions on Robotics.

[53]  Wendy Ju,et al.  Designing robots with movement in mind , 2014, Journal of Human-Robot Interaction.

[54]  Antonio Camurri,et al.  A System for Real-Time Multimodal Analysis of Nonverbal Affective Social Interaction in User-Centric Media , 2010, IEEE Transactions on Multimedia.

[55]  Laurel D. Riek,et al.  A Method for Automatic Detection of Psychomotor Entrainment , 2016, IEEE Transactions on Affective Computing.

[56]  Cynthia Breazeal,et al.  Anticipatory Perceptual Simulation for Human-Robot Joint Practice: Theory and Application Study , 2008, AAAI.

[57]  Siddhartha S. Srinivasa,et al.  Toward seamless human-robot handovers , 2013, Journal of Human-Robot Interaction.

[58]  Laurel D. Riek,et al.  Mobile Robots and Marching Humans: Measuring Synchronous Joint Action While in Motion , 2014, AAAI Fall Symposia.

[59]  Stefanos Nikolaidis,et al.  Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[60]  Maya Cakmak,et al.  Adaptive Coordination Strategies for Human-Robot Handovers , 2015, Robotics: Science and Systems.

[61]  Dmitry Berenson,et al.  Human-robot collaborative manipulation planning using early prediction of human motion , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[62]  Cynthia Breazeal,et al.  Cost-Based Anticipatory Action Selection for Human–Robot Fluency , 2007, IEEE Transactions on Robotics.

[63]  Guy Hoffman,et al.  Synchronization in human-robot Musicianship , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[64]  Larry H. Matthies,et al.  First-Person Activity Recognition: What Are They Doing to Me? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Hema Swetha Koppula,et al.  Anticipatory Planning for Human-Robot Teams , 2014, ISER.

[66]  Julie A. Shah,et al.  Fast target prediction of human reaching motion for cooperative human-robot manipulation tasks using time series classification , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[67]  H. Bekkering,et al.  Joint action: bodies and minds moving together , 2006, Trends in Cognitive Sciences.

[68]  Laurel D. Riek,et al.  Joint action perception to enable fluent human-robot teamwork , 2015, 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[69]  Cynthia Breazeal,et al.  Collaboration in Human-Robot Teams , 2004, AIAA 1st Intelligent Systems Technical Conference.

[70]  Laurel D. Riek,et al.  Detecting and Synthesizing Synchronous Joint Action in Human-Robot Teams , 2015, ICMI.

[71]  Laurel D. Riek,et al.  Coordination Dynamics in Multihuman Multirobot Teams , 2017, IEEE Robotics and Automation Letters.

[72]  Cynthia Breazeal,et al.  Effects of anticipatory action on human-robot teamwork: Efficiency, fluency, and perception of team , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[73]  Sonia Chernova,et al.  Reinforcement Learning from Demonstration through Shaping , 2015, IJCAI.

[74]  T. Vicsek,et al.  Self-organizing processes: The sound of many hands clapping , 2000, Nature.

[75]  C. Frith,et al.  Follow you, Follow me: Continuous Mutual Prediction and Adaptation in Joint Tapping , 2010, Quarterly journal of experimental psychology.

[76]  L. Riek,et al.  Tempo Adaptation and Anticipation Methods for Human-Robot Teams , 2016 .