Pragmatic Frames for Teaching and Learning in Human–Robot Interaction: Review and Challenges

One of the big challenges in robotics today is to learn from human users that are inexperienced in interacting with robots but yet are often used to teach skills flexibly to other humans and to children in particular. A potential route toward natural and efficient learning and teaching in Human-Robot Interaction (HRI) is to leverage the social competences of humans and the underlying interactional mechanisms. In this perspective, this article discusses the importance of pragmatic frames as flexible interaction protocols that provide important contextual cues to enable learners to infer new action or language skills and teachers to convey these cues. After defining and discussing the concept of pragmatic frames, grounded in decades of research in developmental psychology, we study a selection of HRI work in the literature which has focused on learning–teaching interaction and analyze the interactional and learning mechanisms that were used in the light of pragmatic frames. This allows us to show that many of the works have already used in practice, but not always explicitly, basic elements of the pragmatic frames machinery. However, we also show that pragmatic frames have so far been used in a very restricted way as compared to how they are used in human–human interaction and argue that this has been an obstacle preventing robust natural multi-task learning and teaching in HRI. In particular, we explain that two central features of human pragmatic frames, mostly absent of existing HRI studies, are that (1) social peers use rich repertoires of frames, potentially combined together, to convey and infer multiple kinds of cues; (2) new frames can be learnt continually, building on existing ones, and guiding the interaction toward higher levels of complexity and expressivity. To conclude, we give an outlook on the future research direction describing the relevant key challenges that need to be solved for leveraging pragmatic frames for robot learning and teaching.

[1]  David S. Touretzky,et al.  Shaping robot behavior using principles from instrumental conditioning , 1997, Robotics Auton. Syst..

[2]  Manuel Lopes,et al.  Modeling affordances using Bayesian networks , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Katharina Rohlfing,et al.  Learning new words in unfamiliar frames from direct and indirect teaching. , 2013 .

[4]  Ingo Lütkebohle,et al.  The bielefeld anthropomorphic robot head “Flobi” , 2010, 2010 IEEE International Conference on Robotics and Automation.

[5]  Juan Poblete,et al.  Learning words : comparing two-year-olds’ learning success in dyadic and triadic teaching situations embedded in familiar and unfamiliar contexts , 2011 .

[6]  Dongheui Lee,et al.  Incremental kinesthetic teaching of motion primitives using the motion refinement tube , 2011, Auton. Robots.

[7]  Darwin G. Caldwell,et al.  Imitation Learning of Positional and Force Skills Demonstrated via Kinesthetic Teaching and Haptic Input , 2011, Adv. Robotics.

[8]  J. Taylor,et al.  Playing safe? , 1989, Nursing times.

[9]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[10]  Dirk Heylen,et al.  Bridging the Gap between Social Animal and Unsocial Machine: A Survey of Social Signal Processing , 2012, IEEE Transactions on Affective Computing.

[11]  J. Tenenbaum,et al.  Word learning as Bayesian inference. , 2007, Psychological review.

[12]  Andrea Lockerd Thomaz,et al.  Reinforcement Learning with Human Teachers: Evidence of Feedback and Guidance with Implications for Learning Performance , 2006, AAAI.

[13]  Pierre-Yves Oudeyer,et al.  The strategic student approach for life-long exploration and learning , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[14]  Monica N. Nicolescu,et al.  Task Learning through Imitation and Human-robot Interaction , 2005 .

[15]  Britta Wrede,et al.  Pamini: A framework for assembling mixed-initiative human-robot interaction from generic interaction patterns , 2010, SIGDIAL Conference.

[16]  Heriberto Cuayáhuitl,et al.  Robot learning from verbal interaction: A brief survey , 2015 .

[17]  Stefan Kopp,et al.  Generation and Evaluation of Communicative Robot Gesture , 2012, Int. J. Soc. Robotics.

[18]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[19]  M. Tomasello,et al.  What's in a manner of speaking? Children's sensitivity to partner-specific referential precedents. , 2010, Developmental psychology.

[20]  Michael Tomasello,et al.  Three-year-olds understand communicative intentions without language, gestures, or gaze , 2013 .

[21]  Aude Billard,et al.  Teaching a Humanoid Robot to Recognize and Reproduce Social Cues , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[22]  Peter Ford Dominey.  Towards a construction-based framework for development of language, event perception and social cognition: Insights from grounded robotics and simulation , 2007, Neurocomputing.

[23]  Peter Ford Dominey,et al.  The Coordinating Role of Language in Real-Time Multimodal Learning of Cooperative Tasks , 2013, IEEE Transactions on Autonomous Mental Development.

[24]  Philippe Gaussier,et al.  A Robot Learns the Facial Expressions Recognition and Face/Non-face Discrimination Through an Imitation Game , 2014, Int. J. Soc. Robotics.

[25]  Minoru Asada,et al.  A constructive model for the development of joint attention , 2003, Connect. Sci..

[26]  Maya Cakmak,et al.  Learning about objects with human teachers , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  Christian Heath,et al.  IEEE International Symposium on Robot and Human Interactive Communication , 2009 .

[28]  Maya Cakmak,et al.  Power to the People: The Role of Humans in Interactive Machine Learning , 2014, AI Mag..

[29]  J. Iverson,et al.  Investigating motionese: The effect of infant-directed action on infants' attention and object exploration. , 2009, Infant behavior & development.

[30]  Jochen J. Steil,et al.  Interactive imitation learning of object movement skills , 2011, Autonomous Robots.

[31]  M. Matarić,et al.  Task Learning Through Imitation and Human-Robot Interaction , 2004 .

[32]  Giulio Sandini,et al.  The iCub Cognitive Humanoid Robot: An Open-System Research Platform for Enactive Cognition , 2006, 50 Years of Artificial Intelligence.

[33]  Herbert H. Clark,et al.  Grounding in communication , 1991, Perspectives on socially shared cognition.

[34]  Darwin G. Caldwell,et al.  Learning and Reproduction of Gestures by Imitation , 2010, IEEE Robotics & Automation Magazine.

[35]  Leslie Pack Kaelbling,et al.  Effective reinforcement learning for mobile robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[36]  Eiichi Yoshida,et al.  Human-Robot Cooperation Based on Interaction Learning , 2010, From Motor Learning to Interaction Learning in Robots.

[37]  Pierre-Yves Oudeyer,et al.  Active choice of teachers, learning strategies and goals for a socially guided intrinsic motivation learner , 2012, Paladyn J. Behav. Robotics.

[38]  Manuel Lopes,et al.  Affordance-based imitation learning in robots , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  Aude Billard,et al.  Donut as I do: Learning from failed demonstrations , 2011, 2011 IEEE International Conference on Robotics and Automation.

[40]  Nikolaos G. Tsagarakis,et al.  The design of the lower body of the compliant humanoid robot “cCub” , 2011, 2011 IEEE International Conference on Robotics and Automation.

[41]  Oliver Lemon,et al.  Strategic Dialogue Management via Deep Reinforcement Learning , 2015, NIPS 2015.

[42]  Gregory Kuhlmann and Peter Stone and Raymond J. Mooney and Shavlik.  Guiding a Reinforcement Learner with Natural Language Advice: Initial Results in RoboCup Soccer , 2004, AAAI 2004.

[43]  Luc Steels,et al.  Language games for autonomous robots , 2001 .

[44]  Aude Billard,et al.  Incremental learning of gestures by imitation in a humanoid robot , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[45]  Katherine Nelson,et al.  Young Minds in Social Worlds: Experience, Meaning, and Memory , 2007 .

[46]  Chrystopher L. Nehaniv,et al.  Teaching robots by moulding behavior and scaffolding the environment , 2006, HRI '06.

[47]  Bruce Blumberg,et al.  Integrated learning for interactive synthetic characters , 2002, SIGGRAPH.

[48]  Markus Hessinger,et al.  Man-Robot-Cooperation , 2017 .

[49]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[50]  Nabeel Hamdi,et al.  Learning and Communication , 2010 .

[51]  Pierre-Yves Oudeyer,et al.  Active learning of inverse models with intrinsically motivated goal exploration in robots , 2013, Robotics Auton. Syst..

[52]  T. Belpraeme,et al.  Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative Dimensions , 2006 .

[53]  Andrea Lockerd Thomaz,et al.  Tutelage and socially guided robot learning , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[54]  Pierre-Yves Oudeyer,et al.  Robotic clicker training , 2002, Robotics Auton. Syst..

[55]  Sorin Moga,et al.  Learning and communication via imitation: an autonomous robot perspective , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[56]  Pierre-Yves Oudeyer,et al.  An Alternative to Mapping a Word onto a Concept in Language Acquisition: Pragmatic Frames , 2016, Front. Psychol..

[57]  Pierre-Yves Oudeyer,et al.  Studying the co-construction of interaction protocols in collaborative tasks with humans , 2014, 4th International Conference on Development and Learning and on Epigenetic Robotics.

[58]  Henk Nijmeijer,et al.  Robot Programming by Demonstration , 2010, SIMPAR.

[59]  Pierre-Yves Oudeyer,et al.  From Language to Motor Gavagai: Unified Imitation Learning of Multiple Linguistic and Nonlinguistic Sensorimotor Skills , 2013, IEEE Transactions on Autonomous Mental Development.

[60]  Pierre-Yves Oudeyer,et al.  Robot learning simultaneously a task and how to interpret human instructions , 2013, 2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[61]  John Kenneth Salisbury,et al.  Playing it safe [human-friendly robots] , 2004, IEEE Robotics & Automation Magazine.

[62]  Pierre-Yves Oudeyer,et al.  Interactive Learning from Unlabeled Instructions , 2014, UAI.

[63]  J. Bruner.  Child's Talk: Learning to Use Language , 1985 .

[64]  Bruno Maisonnier,et al.  Choregraphe: a graphical tool for humanoid robot programming , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[65]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[66]  Luc Steels,et al.  Aibo''s first words. the social learning of language and meaning. Evolution of Communication , 2002 .

[67]  Jochen J. Steil,et al.  Imitating object movement skills with robots — A task-level approach exploiting generalization and invariance , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[68]  Pradeep K. Khosla,et al.  A Multi-Agent System for Programming Robotic Agents by Human Demonstration , 1998 .

[69]  M. Engelmann.  The Philosophical Investigations , 2013 .

[70]  Peter Stone,et al.  A social reinforcement learning agent , 2001, AGENTS '01.

[71]  Gergely Csibra,et al.  One-Year-Old Infants Appreciate the Referential Nature of Deictic Gestures and Words , 2009, Psychological science.

[72]  Maya Cakmak,et al.  Designing robot learners that ask good questions , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[73]  Guido Bugmann,et al.  Mobile robot programming using natural language , 2002, Robotics Auton. Syst..

[74]  L. Steels,et al.  The social learning of language and meaning , 2001 .

[75]  Pierre-Yves Oudeyer,et al.  Online Optimization of Teaching Sequences with Multi-Armed Bandits , 2014, EDM.

[76]  Catherine M Sandhofer,et al.  More is more: the relationship between vocabulary size and word extension. , 2009, Journal of experimental child psychology.

[77]  Masayuki Inaba,et al.  Learning by watching: extracting reusable task knowledge from visual observation of human performance , 1994, IEEE Trans. Robotics Autom..

[78]  Herwin van Welbergen,et al.  Designing Appropriate Feedback for Virtual Agents and Robots. , 2012, HRI 2012.

[79]  Maya Cakmak,et al.  Keyframe-based Learning from Demonstration , 2012, Int. J. Soc. Robotics.

[80]  C. Breazeal,et al.  Transparency and Socially Guided Machine Learning , 2006 .

[81]  Petra B. Schumacher,et al.  Content and context in incremental processing: “the ham sandwich” revisited , 2014 .

[82]  Philippe Gaussier,et al.  Development of First Social Referencing Skills: Emotional Interaction as a Way to Regulate Robot Behavior , 2014, IEEE Transactions on Autonomous Mental Development.

[83]  Peter Ford Dominey.  Toward a construction-based account of shared intentions in social cognition , 2005, Behavioral and Brain Sciences.

[84]  Wolfgang Prinz,et al.  Rethinking ‘Rational Imitation’ in 14-Month-Old Infants: A Perceptual Distraction Approach , 2012, PloS one.

[85]  Jun Tani,et al.  Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment , 2008, PLoS Comput. Biol..