Survey on frontiers of language and robotics

ABSTRACT The understanding and acquisition of a language in a real-world environment is an important task for future robotics services. Natural language processing and cognitive robotics have both been focusing on the problem for decades using machine learning. However, many problems remain unsolved despite significant progress in machine learning (such as deep learning and probabilistic generative models) during the past decade. The remaining problems have not been systematically surveyed and organized, as most of them are highly interdisciplinary challenges for language and robotics. This study conducts a survey on the frontier of the intersection of the research fields of language and robotics, ranging from logic probabilistic programming to designing a competition to evaluate language understanding systems. We focus on cognitive developmental robots that can learn a language from interaction with their environment and unsupervised learning methods that enable robots to learn a language without hand-crafted training data. GRAPHICAL ABSTRACT

[1]  R. Hepburn,et al.  BEING AND TIME , 2010 .

[2]  Alfred Horn,et al.  On sentences which are true of direct unions of algebras , 1951, Journal of Symbolic Logic.

[3]  J. Austin How to do things with words , 1962 .

[4]  C. Fillmore An Alternative to Checklist Theories of Meaning , 1975 .

[5]  M. Halliday,et al.  Halliday: System and Function in Language : Selected Papers , 1976 .

[6]  M. Halliday Language as social semiotic: The social interpretation of language and meaning , 1976 .

[7]  J. Gibson The Ecological Approach to Visual Perception , 1979 .

[8]  H. Maturana,et al.  Autopoiesis and Cognition : The Realization of the Living (Boston Studies in the Philosophy of Scie , 1980 .

[9]  H. Maturana,et al.  Autopoiesis and Cognition , 1980 .

[10]  G. Lakoff,et al.  Metaphors We Live by , 1982 .

[11]  L. Barsalou,et al.  Ad hoc categories , 1983, Memory & cognition.

[12]  D. Sperber,et al.  Relevance: Communication and Cognition , 1997 .

[13]  Terry Winograd,et al.  Understanding computers and cognition , 1986 .

[14]  Douglas Herrmann,et al.  A Taxonomy of Part-Whole Relations , 1987, Cogn. Sci..

[15]  V. Lifschitz,et al.  The Stable Model Semantics for Logic Programming , 1988, ICLP/SLP.

[16]  G. Lakoff,et al.  Women, Fire, and Dangerous Things: What Categories Reveal about the Mind , 1988 .

[17]  D. Over,et al.  Studies in the Way of Words. , 1989 .

[18]  G. Lakoff Women, fire, and dangerous things : what categories reveal about the mind , 1989 .

[19]  M. Halliday,et al.  Language, Context, and Text: Aspects of Language in a Social-Semiotic Perspective , 1989 .

[20]  Stevan Harnad,et al.  Symbol grounding problem , 1990, Scholarpedia.

[21]  Kenneth A. Ross,et al.  The well-founded semantics for general logic programs , 1991, JACM.

[22]  Stephen C. Levinson,et al.  Rethinking Linguistic Relativity , 1991, Current Anthropology.

[23]  P. Greenfield,et al.  Language, tools and brain: The ontogeny and phylogeny of hierarchically organized sequential behavior , 1991, Behavioral and Brain Sciences.

[24]  Chris Sinha,et al.  Symbol Grounding or the Emergence of Symbols? Vocabulary Growth in Children and a Connectionist Net , 1992 .

[25]  Jerry R. Hobbs,et al.  Interpretation as Abduction , 1993, Artif. Intell..

[26]  Taisuke Sato,et al.  A Statistical Learning Method for Logic Programs with Distribution Semantics , 1995, ICLP.

[27]  G. Rizzolatti,et al.  Premotor cortex and the recognition of motor actions. , 1996, Brain research. Cognitive brain research.

[28]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[29]  S. Muggleton Stochastic Logic Programs , 1996 .

[30]  J. Elman,et al.  Rethinking Innateness: A Connectionist Perspective on Development , 1996 .

[31]  J. Grady Foundations of meaning : primary metaphors and primary scenes , 1997 .

[32]  N. Holland Cognitive linguistics. , 1999, The International journal of psycho-analysis.

[33]  G. Lakoff Philosophy in the flesh , 1999 .

[34]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[35]  Z. Kövecses,et al.  Metaphor and Emotion: Language, Culture, and Body in Human Feeling , 2000 .

[36]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition , 2000, Prentice Hall series in artificial intelligence.

[37]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[38]  G. Fauconnier,et al.  The Way We Think: Conceptual Blending and the Mind''s Hidden Complexities. Basic Books , 2002 .

[39]  Noam Chomsky,et al.  The faculty of language: what is it, who has it, and how did it evolve? , 2002 .

[40]  Mark Steedman,et al.  Generative Models for Statistical Parsing with Combinatory Categorial Grammar , 2002, ACL.

[41]  Pietro Perona,et al.  Object class recognition by unsupervised scale-invariant learning , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[42]  Naoto Iwahashi,et al.  Language acquisition through a human-Crobot interface by combining speech, visual, and behavioral information , 2003, Inf. Sci..

[43]  Tetsuo Ono,et al.  Body Movement Analysis of Human-Robot Interaction , 2003, IJCAI.

[44]  Christopher R. Johnson,et al.  Background to Framenet , 2003 .

[45]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[46]  Naoto Iwahashi A Method for Forming Mutual Beliefs for Communication through Human-robot Multi-modal Interaction , 2003, SIGDIAL Workshop.

[47]  Stefan Wermter,et al.  Towards multimodal neural robot learning , 2004, Robotics Auton. Syst..

[48]  Mark Steedman,et al.  The syntactic process , 2004, Language, speech, and communication.

[49]  R. Gibbs,et al.  Metaphor is grounded in embodied experience , 2004 .

[50]  J. Searle Mind: A Brief Introduction , 2004 .

[51]  Dan Klein,et al.  Corpus-Based Induction of Syntactic Structure: Models of Dependency and Constituency , 2004, ACL.

[52]  Alexander Stoytchev,et al.  Behavior-Grounded Representation of Tool Affordances , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[53]  Michael Gasser,et al.  The Development of Embodied Cognition: Six Lessons from Babies , 2005, Artificial Life.

[54]  M. Tomasello,et al.  Role Reversal Imitation and Language in Typically Developing Infants and Children With Autism , 2005 .

[55]  Jun'ichi Tsujii,et al.  Probabilistic CFG with Latent Annotations , 2005, ACL.

[56]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[57]  Z. Kövecses,et al.  Metaphor in Culture: Universality and Variation , 2005 .

[58]  Alexei A. Efros,et al.  Discovering object categories in image collections , 2005 .

[59]  G. Rizzolatti The mirror neuron system and its function in humans , 2005, Anatomy and Embryology.

[60]  Alexander Stoytchev,et al.  Learning the Affordances of Tools Using a Behavior-Grounded Approach , 2006, Towards Affordance-Based Robot Control.

[61]  Naoto Iwahashi,et al.  Robots That Learn Language: Developmental Approach to Human-Machine Conversations , 2006, EELC.

[62]  Benjamin Kuipers,et al.  Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions , 2006, AAAI.

[63]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[64]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[65]  Jerome A. Feldman,et al.  From Molecule to Metaphor - A Neural Theory of Language , 2006 .

[66]  Tetsuya Ogata,et al.  Experience Based Imitation Using RNNPB , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[67]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[68]  Koji Fujita,et al.  Facing the Logical Problem of Language Evolution (L. Jenkins, Variation and Universals in Biolinguistics ) , 2007 .

[69]  Luc De Raedt,et al.  ProbLog: A Probabilistic Prolog and its Application in Link Discovery , 2007, IJCAI.

[70]  Naoto Iwahashi,et al.  Robots That Learn Language: A Developmental Approach to Situated Human-Robot Conversations , 2007 .

[71]  M. Tomasello Cooperation and Communication in the 2nd Year of Life , 2007 .

[72]  Maya Cakmak,et al.  To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot Control , 2007, Adapt. Behav..

[73]  Dan Klein,et al.  The Infinite PCFG Using Hierarchical Dirichlet Processes , 2007, EMNLP.

[74]  Tomoaki Nakamura,et al.  Multimodal object categorization by a robot , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[75]  Laurent Itti,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Rapid Biologically-inspired Scene Classification Using Features Shared with Visual Attention , 2022 .

[76]  George Loizou,et al.  Computer vision and pattern recognition , 2007, Int. J. Comput. Math..

[77]  René Dirven,et al.  Cognitive English Grammar , 2007 .

[78]  Thomas L. Griffiths,et al.  Bayesian Inference for PCFGs via Markov Chain Monte Carlo , 2007, NAACL.

[79]  Thomas L. Griffiths,et al.  Modeling the effects of memory on human online sentence processing with particle filters , 2008, NIPS.

[80]  Naoto Iwahashi,et al.  Interactive Learning of Spoken Words and Their Meanings Through an Audio-Visual Interface , 2008, IEICE Trans. Inf. Syst..

[81]  Antonio Torralba,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 80 Million Tiny Images: a Large Dataset for Non-parametric Object and Scene Recognition , 2022 .

[82]  Jeff Orkin,et al.  The Restaurant Game: Learning Social Behavior and Language from Thousands of Players Online , 2008, J. Game Dev..

[83]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[84]  L. Steels The symbol grounding problem has been solved, so what’s next? , 2008 .

[85]  Chong Wang,et al.  Simultaneous image classification and annotation , 2009, CVPR.

[86]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[87]  Takayuki Kanda,et al.  Nonverbal leakage in robots: Communication of intentions through seemingly unintentional behavior , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[88]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[89]  Naonori Ueda,et al.  Bayesian Unsupervised Word Segmentation with Nested Pitman-Yor Language Modeling , 2009, ACL.

[90]  Koji Fujita,et al.  A Prospect for Evolutionary Adequacy: Merge and the Evolution and Development of Human Language , 2009, Biolinguistics.

[91]  Takayuki Kanda,et al.  Providing route directions: Design of robot's utterance, gesture, and timing , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[92]  Hoifung Poon,et al.  Unsupervised Semantic Parsing , 2009, EMNLP.

[93]  Mark Johnson,et al.  Improving Unsupervised Dependency Parsing with Richer Contexts and Smoothing , 2009, NAACL.

[94]  Phil Blunsom,et al.  Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics , 2009 .

[95]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[96]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[97]  Antonio Torralba,et al.  Recognizing indoor scenes , 2009, CVPR.

[98]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[99]  Satoshi Nakamura,et al.  Object Manipulation Dialogue by Estimating Utterance Understanding Probability in a Robot Language Acquisition Framework , 2010 .

[100]  Takayuki Nagai,et al.  Robots that Learn to Communicate: A Developmental Approach to Personally and Physically Situated Human-Robot Conversations , 2010, AAAI Fall Symposium: Dialog with Robots.

[101]  Yuichiro Yoshikawa,et al.  Simulator platform that enables social interaction simulation — SIGVerse: SocioIntelliGenesis simulator , 2010, 2010 IEEE/SICE International Symposium on System Integration.

[102]  Tomoaki Nakamura,et al.  Forming Object Concept Using Bayesian Network , 2010 .

[103]  Tetsuya Ogata,et al.  Inter-modality mapping in robot with recurrent neural network , 2010, Pattern Recognit. Lett..

[104]  Valentin I. Spitkovsky,et al.  From Baby Steps to Leapfrog: How “Less is More” in Unsupervised Dependency Parsing , 2010, NAACL.

[105]  Thomas L. Griffiths,et al.  The nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies , 2007, JACM.

[106]  Krista A. Ehinger,et al.  SUN database: Large-scale scene recognition from abbey to zoo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[107]  Danijel Skocaj,et al.  Self-supervised cross-modal online learning of basic object affordances for developmental robotic systems , 2010, 2010 IEEE International Conference on Robotics and Automation.

[108]  Gerard J. Steen,et al.  A method for linguistic metaphor identification : from MIP to MIPVU , 2010 .

[109]  Andreas Krause,et al.  Discriminative Clustering by Regularized Information Maximization , 2010, NIPS.

[110]  Hiroshi G. Okuno,et al.  Design and Implementation of Robot Audition System 'HARK' — Open Source Software for Listening to Three Simultaneous Speakers , 2010, Adv. Robotics.

[111]  Tadahiro Taniguchi,et al.  Double articulation analyzer for unsegmented human motion using Pitman-Yor language model and infinite hidden Markov model , 2011, 2011 IEEE/SICE International Symposium on System Integration (SII).

[112]  Satoshi Nakamura,et al.  Situated Spoken Dialogue with Robots Using Active Learning , 2011, Adv. Robotics.

[113]  Alberto Maria Segre,et al.  The Use of Twitter to Track Levels of Disease Activity and Public Concern in the U.S. during the Influenza A H1N1 Pandemic , 2011, PloS one.

[114]  James M. Rehg,et al.  CENTRIST: A Visual Descriptor for Scene Categorization , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[115]  Luca Iocchi,et al.  RoboCup@Home: Adaptive Benchmarking of Robot Bodies and Minds , 2011, ICSR.

[116]  Tomoaki Nakamura,et al.  Grounding of Word Meanings in Latent Dirichlet Allocation-Based Multimodal Concepts , 2011, Adv. Robotics.

[117]  Nathan Schneider,et al.  Association for Computational Linguistics: Human Language Technologies , 2011 .

[118]  Hiroyuki Shindo,et al.  Bayesian Symbol-Refined Tree Substitution Grammars for Syntactic Parsing , 2012, ACL.

[119]  Tomoki Toda,et al.  An Extended Mobile Manipulation Robot Learning Novel Objects , 2012 .

[120]  Zhuowen Tu,et al.  Unsupervised object class discovery via saliency-guided multiple class learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[121]  R. Amant,et al.  Affordances for robots: a brief survey , 2012 .

[122]  B. Bergen Louder Than Words: The New Science of How the Mind Makes Meaning , 2012 .

[123]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[124]  Tomoki Toda,et al.  Learning Novel Objects for Extended Mobile Manipulation , 2012, J. Intell. Robotic Syst..

[125]  Tomoaki Nakamura,et al.  Online learning of concepts and words using multimodal LDA and hierarchical Pitman-Yor Language Model , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[126]  Bernt Schiele,et al.  Grounding Action Descriptions in Videos , 2013, TACL.

[127]  Olivier Mangin,et al.  Learning semantic components from subsymbolic multimodal perception , 2013, 2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[128]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[129]  Mark Steedman,et al.  Combined Distributional and Logical Semantics , 2013, TACL.

[130]  Yoshiki Ando,et al.  Formation of hierarchical object concept using hierarchical latent Dirichlet allocation , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[131]  Peter Ford Dominey,et al.  Multi-modal convergence maps: from body schema and self-representation to mental imagery , 2013, Adapt. Behav..

[132]  Xinlei Chen,et al.  NEIL: Extracting Visual Knowledge from Web Data , 2013, 2013 IEEE International Conference on Computer Vision.

[133]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[134]  Cynthia Breazeal,et al.  Crowdsourcing human-robot interaction , 2013, HRI 2013.

[135]  Sinan Kalkan,et al.  The learning of adjectives and nouns from affordance and appearance features , 2013, Adapt. Behav..

[136]  Dick Wilkinson Concise Thesaurus of Traditional English Metaphors , 2013 .

[137]  Hoifung Poon,et al.  Grounded Unsupervised Semantic Parsing , 2013, ACL.

[138]  Yonatan Bisk,et al.  An HDP Model for Inducing Combinatory Categorial Grammars , 2013, TACL.

[139]  Quoc V. Le,et al.  Grounded Compositional Semantics for Finding and Describing Images with Sentences , 2014, TACL.

[140]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[141]  Ruslan Salakhutdinov,et al.  Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models , 2014, ArXiv.

[142]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[143]  Tomoaki Nakamura,et al.  Mutual learning of an object concept and language model based on MLDA and NPYLM , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[144]  Bernt Schiele,et al.  Coherent Multi-sentence Video Description with Variable Level of Detail , 2014, GCPR.

[145]  F. Pulvermüller The syntax of action , 2014, Trends in Cognitive Sciences.

[146]  Guy Dove,et al.  Thinking in Words: Language as an Embodied Medium of Thought , 2014, Top. Cogn. Sci..

[147]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[148]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[149]  Luc De Raedt,et al.  Inference and learning in probabilistic logic programs using weighted Boolean formulas , 2013, Theory and Practice of Logic Programming.

[150]  Tetsuya Ogata,et al.  Audio-visual speech recognition using deep learning , 2014, Applied Intelligence.

[151]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[152]  Andrea Lockerd Thomaz,et al.  Robot Learning from Human Teachers , 2014, Robot Learning from Human Teachers.

[153]  Robinson Piramuthu,et al.  HD-CNN: Hierarchical Deep Convolutional Neural Networks for Large Scale Visual Recognition , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[154]  Wataru Takano Learning motion primitives and annotative texts from crowd-sourcing , 2015 .

[155]  Pierre-Yves Oudeyer,et al.  MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization , 2015, PloS one.

[156]  Christopher Potts,et al.  Learning Distributed Word Representations for Natural Logic Reasoning , 2014, AAAI Spring Symposia.

[157]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[158]  Yuxin Chen,et al.  Cross-situational noun and adjective learning in an interactive scenario , 2015, 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[159]  Nikolaos Mavridis,et al.  A review of verbal and non-verbal human-robot interactive communication , 2014, Robotics Auton. Syst..

[160]  Antonios Gasteratos,et al.  Semantic mapping for mobile robotics tasks: A survey , 2015, Robotics Auton. Syst..

[161]  Sanja Fidler,et al.  Skip-Thought Vectors , 2015, NIPS.

[162]  Margaret Mitchell,et al.  VQA: Visual Question Answering , 2015, International Journal of Computer Vision.

[163]  Joelle Pineau,et al.  Incorporating Unstructured Textual Knowledge Sources into Neural Dialogue Systems , 2015 .

[164]  Tae-Kyun Kim,et al.  STARE: Spatio-Temporal Attention Relocation for Multiple Structured Activities Detection , 2015, IEEE Transactions on Image Processing.

[165]  Samy Bengio,et al.  Show and tell: A neural image caption generator , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[166]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[167]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[168]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[169]  Fei-Fei Li,et al.  Deep visual-semantic alignments for generating image descriptions , 2015, CVPR.

[170]  Alexander M. Rush,et al.  Transforming Dependencies into Phrase Structures , 2015, NAACL.

[171]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[172]  Jin-Hui Zhu,et al.  Affordance Research in Developmental Robotics: A Survey , 2016, IEEE Transactions on Cognitive and Developmental Systems.

[173]  Jean Oh,et al.  Attention-based Multimodal Neural Machine Translation , 2016, WMT.

[174]  Sanja Fidler,et al.  MovieQA: Understanding Stories in Movies through Question-Answering , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[175]  Tomoaki Nakamura,et al.  Learning word meanings and grammar for verbalization of daily life activities using multilayered multimodal latent Dirichlet allocation and Bayesian hidden Markov models , 2016, Adv. Robotics.

[176]  Ali Farhadi,et al.  Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding , 2016, ECCV.

[177]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[178]  William W. Cohen TensorLog: A Differentiable Deductive Database , 2016, ArXiv.

[179]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[180]  Ashutosh Modi,et al.  Event Embeddings for Semantic Script Modeling , 2016, CoNLL.

[181]  Pascual Martínez-Gómez,et al.  ccg2lambda: A Compositional Semantics System , 2016, ACL.

[182]  Juliane Hahn,et al.  Halliday System And Function In Language Selected Papers , 2016 .

[183]  Naoaki Okazaki,et al.  Learning Semantically and Additively Compositional Distributional Representations , 2016, ACL.

[184]  Matthew R. Walter,et al.  Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences , 2015, AAAI.

[185]  Tadahiro Taniguchi,et al.  Spatial Concept Acquisition for a Mobile Robot That Integrates Self-Localization and Unsupervised Word Discovery From Spoken Sentences , 2016, IEEE Transactions on Cognitive and Developmental Systems.

[186]  Barbara Mayer,et al.  Concept Image And Symbol The Cognitive Basis Of Grammar , 2016 .

[187]  William Yang Wang,et al.  Learning First-Order Logic Embeddings via Matrix Factorization , 2016, IJCAI.

[188]  Miriam R. L. Petruck Introduction to MetaNet , 2016 .

[189]  Kewei Tu,et al.  Unsupervised Neural Dependency Parsing , 2016, EMNLP.

[190]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[191]  Apostol Natsev,et al.  YouTube-8M: A Large-Scale Video Classification Benchmark , 2016, ArXiv.

[192]  Tadahiro Taniguchi,et al.  Nonparametric Bayesian Double Articulation Analyzer for Direct Language Acquisition From Continuous Speech Signals , 2015, IEEE Transactions on Cognitive and Developmental Systems.

[193]  Tomoaki Nakamura,et al.  Symbol emergence in robotics: a survey , 2015, Adv. Robotics.

[194]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[195]  Mark Johnson,et al.  Grammar induction from (lots of) words alone , 2016, COLING.

[196]  Tamim Asfour,et al.  The KIT Motion-Language Dataset , 2016, Big Data.

[197]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[198]  Thomas A. Funkhouser,et al.  MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments , 2017, ArXiv.

[199]  Demis Hassabis,et al.  Grounded Language Learning in a Simulated 3D World , 2017, ArXiv.

[200]  Yoshiaki Mizuchi,et al.  Cloud-based multimodal human-robot interaction simulator utilizing ROS and unity frameworks , 2017, 2017 IEEE/SICE International Symposium on System Integration (SII).

[201]  牧野 成一,et al.  日英共通メタファー辞典 = A bilingual dictionary of English and Japanese metaphors , 2017 .

[202]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[203]  Athanasios S. Polydoros,et al.  Survey of Model-Based Reinforcement Learning: Applications on Robotics , 2017, J. Intell. Robotic Syst..

[204]  Marc Peter Deisenroth,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[205]  Stephen H. Bach,et al.  Hinge-Loss Markov Random Fields and Probabilistic Soft Logic , 2015, J. Mach. Learn. Res..

[206]  Tim Rocktäschel,et al.  End-to-end Differentiable Proving , 2017, NIPS.

[207]  Tomoaki Nakamura,et al.  Online Algorithm for Robots to Learn Object Concepts and Language Model , 2017, IEEE Transactions on Cognitive and Developmental Systems.

[208]  Dumitru Erhan,et al.  Show and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[209]  Tadahiro Taniguchi,et al.  Online spatial concept and lexical acquisition with simultaneous localization and mapping , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[210]  Masahide Kaneko,et al.  Segmenting Continuous Motions with Hidden Semi-markov Models and Gaussian Processes , 2017, Front. Neurorobot..

[211]  Ali Farhadi,et al.  AI2-THOR: An Interactive 3D Environment for Visual AI , 2017, ArXiv.

[212]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[213]  Yu Liu,et al.  CNN-RNN: a large-scale hierarchical image classification framework , 2018, Multimedia Tools and Applications.

[214]  Pascual Martínez-Gómez,et al.  Determining Semantic Textual Similarity using Natural Deduction Proofs , 2017, EMNLP.

[215]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[216]  Holger Schwenk,et al.  Supervised Learning of Universal Sentence Representations from Natural Language Inference Data , 2017, EMNLP.

[217]  B. Scassellati,et al.  Social eye gaze in human-robot interaction , 2017, J. Hum. Robot Interact..

[218]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[219]  Angelo Cangelosi,et al.  A review of abstract concept learning in embodied agents and robots , 2018, Philosophical Transactions of the Royal Society B: Biological Sciences.

[220]  Angelo Cangelosi,et al.  Affordances in Psychology, Neuroscience, and Robotics: A Survey , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[221]  Tomoaki Nakamura,et al.  SERKET: An Architecture for Connecting Stochastic Models to Realize a Large-Scale Cognitive Model , 2017, Front. Neurorobot..

[222]  Niranjan Balasubramanian,et al.  Event Representations with Tensor-based Compositions , 2017, AAAI.

[223]  A. Utsumi A Distributional Semantic Model of Visually Indirect Grounding for Abstract Words , 2018 .

[224]  Jason Weston,et al.  Talk the Walk: Navigating New York City through Grounded Dialogue , 2018, ArXiv.

[225]  Stefan Lee,et al.  Embodied Question Answering , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[226]  Qi Wu,et al.  Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[227]  Peter Stone,et al.  Multi-modal Predicate Identification using Dynamically Learned Robot Controllers , 2018, IJCAI.

[228]  Peter Stone,et al.  Guiding Exploratory Behaviors for Multi-Modal Grounding of Linguistic Descriptions , 2018, AAAI.

[229]  Daichi Mochihashi,et al.  A Probabilistic Approach to Unsupervised Induction of Combinatory Categorial Grammar in Situated Human-Robot Interaction , 2018, 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids).

[230]  Shuo Yang,et al.  Faceness-Net: Face Detection through Deep Facial Part Responses , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[231]  Tamim Asfour,et al.  Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networks , 2017, Robotics Auton. Syst..

[232]  Tadahiro Taniguchi,et al.  Hierarchical Spatial Concept Formation Based on Multimodal Information for Human Support Robots , 2018, Front. Neurorobot..

[233]  Simon Brodeur,et al.  HoME: a Household Multimodal Environment , 2017, ICLR.

[234]  Kuniyuki Takahashi,et al.  Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructions , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[235]  Tetsuya Ogata,et al.  Paired Recurrent Autoencoders for Bidirectional Translation Between Robot Actions and Linguistic Descriptions , 2018, IEEE Robotics and Automation Letters.

[236]  Hiroyuki Okada,et al.  Semantic reasoning in service robots using expert systems , 2019, Robotics Auton. Syst..

[237]  Joelle Pineau,et al.  The Second Conversational Intelligence Challenge (ConvAI2) , 2019, The NeurIPS '18 Competition.

[238]  Yoshiaki Mizuchi,et al.  Robot Competition to Evaluate Guidance Skill for General Users in VR Environment , 2019, 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[239]  Y-Lan Boureau,et al.  Overview of the sixth dialog system technology challenge: DSTC6 , 2019, Comput. Speech Lang..

[240]  Lokendra Shastri,et al.  A Connectionist model of Planning via Back-chaining Search , 2019, Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science Society.

[241]  Florentin Wörgötter,et al.  Symbol Emergence in Cognitive Developmental Systems: A Survey , 2018, IEEE Transactions on Cognitive and Developmental Systems.