Robot Self-Assessment and Expression: A Learning Framework

Future autonomous robots that operate in teams with humans should have capabilities that facilitate intuitive and/or implicit communication, for example in the form of emotional expressions. These emotional expressions should be presented clearly to the human to promote adequate understanding of robot behaviors and intent. In this paper, we present a Robot Self-Assessment and Expression framework derived from reinforcement theory of motivation and the current state-of-the-art in machine learning. The proposed framework theoretically describes how a robot could display emotional expressions depending on both predicted outcomes and actual outcomes of a task. The end goal for this framework design will be for the robot to obtain anticipatory guidance and performance feedback from a human instructor during a training task. Future research and areas for testing and validation of the framework are discussed.

[1]  Cynthia Breazeal,et al.  Emotion and sociable humanoid robots , 2003, Int. J. Hum. Comput. Stud..

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[4]  Roderic A. Grupen,et al.  A feedback control structure for on-line learning tasks , 1997, Robotics Auton. Syst..

[5]  Florian Jentsch,et al.  The importance of shared mental models and shared situation awareness for transforming robots from tools to teammates , 2012, Defense, Security, and Sensing.

[6]  Florian Jentsch,et al.  A Research Approach to Shared Mental Models and Situation Assessment in Future Robot Teams , 2011 .

[7]  Pierre-Yves Oudeyer,et al.  How Evolution May Work Through Curiosity-Driven Developmental Process , 2016, Top. Cogn. Sci..

[8]  W. Timberlake.  Behavior systems, associationism, and Pavlovian conditioning , 1994, Psychonomic bulletin & review.

[9]  Florian Jentsch,et al.  Developing a Tactical Language for Future Robotic Teammates , 2013 .

[10]  Andrea Lockerd Thomaz,et al.  Teaching and working with robots as a collaboration , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[11]  Claire Tourmen.  With or Beyond Piaget? A Dialogue between New Probabilistic Models of Learning and the Theories of Jean Piaget , 2016, Human Development.

[12]  Heriberto Cuayáhuitl,et al.  Training an Interactive Humanoid Robot Using Multimodal Deep Reinforcement Learning , 2016, ArXiv.

[13]  Terrence Fong,et al.  Robot, asker of questions , 2003, Robotics Auton. Syst..

[14]  Andrea Lockerd Thomaz,et al.  Reinforcement Learning with Human Teachers: Evidence of Feedback and Guidance with Implications for Learning Performance , 2006, AAAI.

[15]  B. Skinner.  Operant Behavior , 2021, Encyclopedia of Evolutionary Psychological Science.

[16]  Brian Scassellati,et al.  Infant-like Social Interactions between a Robot and a Human Caregiver , 2000, Adapt. Behav..

[17]  Michael A. Goodrich,et al.  Towards predicting robot team performance , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[18]  Claudia Wagner,et al.  Anthropomorphic inferences from emotional nonverbal cues: A case study , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[19]  R. Mayer.  Should there be a three-strikes rule against pure discovery learning? The case for guided methods of instruction. , 2004, The American psychologist.

[20]  Ross A. Knepper,et al.  Implicit Communication in a Joint Action , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[21]  Maya Cakmak,et al.  Transparent active learning for robots , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[22]  Jessie Y. C. Chen,et al.  Human-robot interaction: Developing trust in robots , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[23]  Andrea Lockerd Thomaz,et al.  Tutelage and socially guided robot learning , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[24]  R. Armstrong.  Review of Research. , 1990 .

[25]  P N Johnson-Laird,et al.  Deductive reasoning. , 1999, Annual review of psychology.

[26]  Cynthia Breazeal,et al.  Real-Time Interactive Reinforcement Learning for Robots , 2005 .

[27]  Cynthia Breazeal,et al.  Toward sociable robots , 2003, Robotics Auton. Syst..

[28]  Peter Stone,et al.  Deep Reinforcement Learning in Parameterized Action Space , 2015, ICLR.

[29]  C. Breazeal,et al.  Experiments in socially guided exploration: lessons learned in building robots that learn with and without human teachers , 2008, Connect. Sci..

[30]  Illah R. Nourbakhsh,et al.  The role of expressiveness and attention in human-robot interaction , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[31]  K. Holyoak,et al.  Analogical problem solving , 1980, Cognitive Psychology.

[32]  H. Lövheim.  A new three-dimensional model for emotions and monoamine neurotransmitters. , 2012, Medical hypotheses.

[33]  E. Salas,et al.  Team cognition : understanding the factors that drive process and performance , 2004 .

[34]  Matthias Scheutz,et al.  The utility of affect expression in natural language interactions in joint human-robot tasks , 2006, HRI '06.

[35]  Jean Piaget.  Piaget’s Theory , 1976 .

[36]  Cynthia Breazeal,et al.  Social interactions in HRI: the robot view , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[37]  Cynthia Breazeal.  Emotive qualities in robot speech , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[38]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[39]  Evan Heit,et al.  Relations between inductive reasoning and deductive reasoning. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[40]  Naomi J. Aldrich,et al.  Does Discovery-Based Instruction Enhance Learning?. , 2011 .

[41]  Zoran Popovic,et al.  Where to Add Actions in Human-in-the-Loop Reinforcement Learning , 2017, AAAI.