Knowledge Processing for Autonomous Robots

Autonomous robots are becoming more and more skilled in performing human-scale manipulation tasks, and will soon become common co-workers in our homes. In this work, we describe a framework for representing the knowledge that an autonomous robot needs for performing household tasks, for automated reasoning on this knowledge, and for acquiring new knowledge from sources on the Internet and from observations of human activities. The developed methods allow programmers to write more general and flexible control programs and help robots to execute incomplete and underspecified instructions.

[1]  Hugo Liu,et al.  ConceptNet — A Practical Commonsense Reasoning Tool-Kit , 2004 .

[2]  Michael Beetz,et al.  Using Physics- and Sensor-based Simulation for High-Fidelity Temporal Projection of Realistic Robot Behavior , 2009, ICAPS.

[3]  Raffaello D'Andrea,et al.  RoboEarth: connecting robots worldwide , 2009, ICIS.

[4]  Michael Beetz,et al.  Generality and legibility in mobile manipulation , 2010, Auton. Robots.

[5]  Il Hong Suh,et al.  Ontology-based multi-layered robot knowledge framework (OMRKF) for robot intelligence , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Geoffrey A. Hollinger,et al.  HERB: a home exploring robotic butler , 2010, Auton. Robots.

[7]  A. T. Schreiber,et al.  Prolog-based infrastructure for RDF: performance and scalability , 2003 .

[8]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[9]  Ian Horrocks,et al.  Description Logics , 2008, Handbook of Knowledge Representation.

[10]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[11]  Aaron F. Bobick,et al.  Recognition of Visual Activities and Interactions by Stochastic Parsing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Moritz Tenorth,et al.  RoboEarth - A World Wide Web for Robots , 2011, ICRA 2011.

[13]  Gottlob Frege,et al.  Begriffsschrift, eine der arithmetischen nachgebildete Formelsprache des reinen Denkens , 1879 .

[14]  Adam Pease,et al.  Towards a standard upper ontology , 2001, FOIS.

[15]  Yarden Katz,et al.  Pellet: A practical OWL-DL reasoner , 2007, J. Web Semant..

[16]  Luc De Raedt,et al.  Relational Transformation-based Tagging for Activity Recognition , 2008, Fundam. Informaticae.

[17]  Rachid Alami,et al.  Which one? Grounding the referent based on efficient human-robot interaction , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[18]  Kathryn B. Laskey.  MEBN: A language for first-order Bayesian knowledge bases , 2008, Artif. Intell..

[19]  Daniel S. Weld,et al.  UCPOP: A Sound, Complete, Partial Order Planner for ADL , 1992, KR.

[20]  Douglas B. Lenat,et al.  CYC: a large-scale investment in knowledge infrastructure , 1995, CACM.

[21]  Sridevi Parise,et al.  Unsupervised Learning with Permuted Data , 2003, ICML.

[22]  Larry S. Davis,et al.  Understanding videos, constructing plots learning a visually grounded storyline model from annotated videos , 2009, CVPR.

[23]  Push Singh,et al.  LifeNet: A Propositional Model of Ordinary Human Activity , 2003 .

[24]  Dana Kulic,et al.  Online Segmentation and Clustering From Continuous Observation of Whole Body Motions , 2009, IEEE Transactions on Robotics.

[25]  Takahiro Kawamura,et al.  Semantic Matching of Web Services Capabilities , 2002, SEMWEB.

[26]  Nassir Navab,et al.  Workflow monitoring based on 3D motion features , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[27]  Henry A. Kautz,et al.  Generalized Plan Recognition , 1986, AAAI.

[28]  Matthias Jüngel,et al.  XABSL - A Pragmatic Approach to Behavior Engineering , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Silvia Coradeschi,et al.  Grounding commonsense knowledge in intelligent systems , 2009, J. Ambient Intell. Smart Environ..

[30]  Scott E. Fahlman,et al.  Marker-Passing Inference in the Scone Knowledge-Base System , 2006, KSEM.

[31]  Ben Taskar,et al.  Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning) , 2007 .

[32]  Jake K. Aggarwal,et al.  Recognition of Composite Human Activities through Context-Free Grammar Based Representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[33]  Dejan Pangercic,et al.  Combining perception and knowledge processing for everyday manipulation , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  Michael Beetz,et al.  Accurate Human Motion Capture Using an Ergonomics-Based Anthropometric Human Model , 2008, AMDO.

[35]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[36]  Manuela M. Veloso,et al.  Conditional random fields for activity recognition , 2007, AAMAS '07.

[37]  David C. Minnen,et al.  Propagation networks for recognition of partially ordered sequential action , 2004, CVPR 2004.

[38]  Moritz Tenorth,et al.  Towards Automated Models of Activities of Daily Life , 2009 .

[39]  Remi Arnaud,et al.  COLLADA: Sailing the Gulf of 3D Digital Content Creation , 2006 .

[40]  Tamim Asfour,et al.  Toward an Unified Representation for Imitation of Human Motion on Humanoids , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[41]  Christian Bauckhage,et al.  Action Recognition in Videos Using Nonnegative Tensor Factorization , 2010, 2010 20th International Conference on Pattern Recognition.

[42]  Gerhard Weikum,et al.  WWW 2007 / Track: Semantic Web Session: Ontologies ABSTRACT YAGO: A Core of Semantic Knowledge , 2022 .

[43]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.

[44]  Irfan A. Essa,et al.  Structure from Statistics - Unsupervised Activity Analysis using Suffix Trees , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[45]  Moritz Tenorth,et al.  KNOWROB — knowledge processing for autonomous personal robots , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[46]  Henry A. Kautz,et al.  Fine-grained activity recognition by aggregating abstract object usage , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[47]  Chris H. Q. Ding,et al.  Adaptive dimension reduction for clustering high dimensional data , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[48]  Moritz Tenorth,et al.  How-models of human reaching movements in the context of everyday manipulation activities , 2011, 2011 IEEE International Conference on Robotics and Automation.

[49]  Eric L. Sauser,et al.  An Approach Based on Hidden Markov Model and Gaussian Mixture Regression , 2010 .

[50]  Nico Blodow,et al.  Inferring Generalized Pick-and-Place Tasks from Pointing Gestures , 2011, ICRA 2011.

[51]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[52]  Moritz Tenorth,et al.  Learning organizational principles in human environments , 2012, 2012 IEEE International Conference on Robotics and Automation.

[53]  Rémi Ronfard,et al.  Free viewpoint action recognition using motion history volumes , 2006, Comput. Vis. Image Underst..

[54]  P. D. O'Brien,et al.  FIPA — Towards a Standard for Software Agents , 1998 .

[55]  Il Hong Suh,et al.  Ontology-Based Unified Robot Knowledge for Service Robots in Indoor Environments , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[56]  Stefan Holzer,et al.  Towards autonomous robotic butlers: Lessons learned with the PR2 , 2011, 2011 IEEE International Conference on Robotics and Automation.

[57]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[58]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[59]  Deb Roy,et al.  Grounded Situation Models for Robots: Where words and percepts meet , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[60]  Dejan Pangercic,et al.  Real-time CAD model matching for mobile manipulation and grasping , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[61]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[62]  Stefan Schaal,et al.  Learning and generalization of motor skills by learning from demonstration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[63]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[64]  J.-P. Laumond,et al.  Move3D: A generic platform for path planning , 2001, Proceedings of the 2001 IEEE International Symposium on Assembly and Task Planning (ISATP2001). Assembly and Disassembly in the Twenty-first Century. (Cat. No.01TH8560).

[65]  Robert M. Fung,et al.  Backward Simulation in Bayesian Networks , 1994, UAI.

[66]  Michael Beetz,et al.  Compact models of motor primitive variations for predictable reaching and obstacle avoidance , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[67]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[68]  Michael Thielscher,et al.  Representing the Knowledge of a Robot , 2000, KR.

[69]  Alexander Verl,et al.  Care-O-bot® 3 - creating a product vision for service robot applications by integrating design and technology , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[70]  Kenneth D. Forbus.  Qualitative Process Theory , 1984, Artificial Intelligence.

[71]  Drew McDermott,et al.  Robot Planning , 1991, AI Mag..

[72]  Boris Motik,et al.  OWL 2 Web Ontology Language: structural specification and functional-style syntax , 2008 .

[73]  Kenneth C. Arnold,et al.  Learning Hierarchical Plans by Reading Simple English Narratives , 2009 .

[74]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[75]  Jessica K. Hodgins,et al.  Guide to the Carnegie Mellon University Multimodal Activity (CMU-MMAC) Database , 2008 .

[76]  Trevor Darrell,et al.  Conditional Random Fields for Object Recognition , 2004, NIPS.

[77]  Moritz Tenorth,et al.  Understanding and executing instructions for everyday manipulation tasks from the World Wide Web , 2010, 2010 IEEE International Conference on Robotics and Automation.

[78]  Kenneth D. Forbus.  Introducing Actions into Qualitative Simulation , 1989, IJCAI.

[79]  James A. Hendler,et al.  The Semantic Web 10 , 2011 .

[80]  Michael Beetz,et al.  Parameterizing actions to have the appropriate effects , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[81]  Matthai Philipose,et al.  Mining models of human activities from the web , 2004, WWW '04.

[82]  Nico Blodow,et al.  Perception and probabilistic anchoring for dynamic world state logging , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[83]  James A. Hendler,et al.  HTN Planning: Complexity and Expressivity , 1994, AAAI.

[84]  Rohit J. Kate,et al.  Learning to Transform Natural to Formal Languages , 2005, AAAI.

[85]  Michael J. Witbrock,et al.  An Introduction to the Syntax and Content of Cyc , 2006, AAAI Spring Symposium: Formalizing and Compiling Background Knowledge and Its Applications to Knowledge Representation and Question Answering.

[86]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[87]  Henry Kautz,et al.  Hierarchical recognition of activities of daily living using multi-scale, multi-perspective vision and RFID , 2008 .

[88]  Daniel S. Weld,et al.  Autonomously semantifying wikipedia , 2007, CIKM '07.

[89]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[90]  Michael Beetz,et al.  Simulation-based temporal projection of everyday robot object manipulation , 2011, AAMAS.

[91]  Michael Beetz,et al.  ORO, a knowledge management module for cognitive architectures in robotics , 2010, IROS 2010.

[92]  Christopher W. Geib,et al.  The meaning of action: a review on action recognition and mapping , 2007, Adv. Robotics.

[93]  John S. Zelek,et al.  Human-robot interaction with minimal spanning natural language template for autonomous and tele-operated control , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[94]  Michael Beetz,et al.  GrAM: Reasoning with Grounded Action Models by Combining Knowledge Representation and Data Mining , 2006, Towards Affordance-Based Robot Control.

[95]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[96]  Michael Beetz,et al.  Tracking humans interacting with the environment using efficient hierarchical sampling and layered observation models , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[97]  Moritz Tenorth,et al.  Putting People's Common Sense into Knowledge Bases of Household Robots , 2010, KI.

[98]  Nico Blodow,et al.  The Assistive Kitchen — A demonstration scenario for cognitive technical systems , 2007, RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication.

[99]  Boris Motik,et al.  HermiT: A Highly-Efficient OWL Reasoner , 2008, OWLED.

[100]  Moritz Tenorth,et al.  KNOWROB-MAP - knowledge-linked semantic object maps , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[101]  T. Shallice,et al.  The simulation of action disorganisation in complex activities of daily living , 2005, Cognitive neuropsychology.

[102]  Moritz Tenorth,et al.  The TUM Kitchen Data Set of everyday manipulation activities for motion tracking and action recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[103]  Svetha Venkatesh,et al.  Recognition of human activity through hierarchical stochastic learning , 2003, Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003. (PerCom 2003)..

[104]  Michael Thielscher,et al.  Introduction to the Fluent Calculus , 1998, Electron. Trans. Artif. Intell..

[105]  Svetha Venkatesh,et al.  The Hidden Permutation Model and Location-Based Activity Recognition , 2008, AAAI.

[106]  Moritz Tenorth,et al.  CRAM — A Cognitive Robot Abstract Machine for everyday manipulation in human environments , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[107]  Erik T. Mueller,et al.  Open Mind Common Sense: Knowledge Acquisition from the General Public , 2002, OTM.

[108]  Michael Beetz,et al.  Towards semantic robot description languages , 2011, 2011 IEEE International Conference on Robotics and Automation.

[109]  Stefanie Tellex,et al.  Spatial routines for a simulated speech-controlled vehicle , 2006, HRI '06.

[110]  Leon Sterling,et al.  The Art of Prolog - Advanced Programming Techniques , 1986 .

[111]  Michael Beetz,et al.  Bayesian Logic Networks , 2009 .

[112]  Oren Etzioni,et al.  The Tradeoffs Between Open and Traditional Relation Extraction , 2008, ACL.

[113]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[114]  Diego Calvanese,et al.  The Description Logic Handbook , 2007 .

[115]  Rainer Drath,et al.  AutomationML - the glue for seamless automation engineering , 2008, 2008 IEEE International Conference on Emerging Technologies and Factory Automation.

[116]  Jan Wielemaker,et al.  Processing OWL2 Ontologies using Thea: An Application of Logic Programming , 2009, OWLED.

[117]  Fernando De la Torre,et al.  Canonical Time Warping for Alignment of Human Behavior , 2009, NIPS.

[118]  Robert P. Goldman,et al.  A New Model of Plan Recognition , 1999, UAI.

[119]  Yoav Shoham,et al.  Ten requirements for a theory of change , 1986, New Generation Computing.

[120]  G. Niemeyer,et al.  Springer Handbook of Robotics: Chapter 31 , 2008 .

[121]  Alcherio Martinoli,et al.  A quantitative method for comparing trajectories of mobile robots using point distribution models , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.