A temporal logic-based planning and execution monitoring framework for unmanned aircraft systems

Research with autonomous unmanned aircraft systems is reaching a new degree of sophistication where targeted missions require complex types of deliberative capability integrated in a practical manner in such systems. Due to these pragmatic constraints, integration is just as important as theoretical and applied work in developing the actual deliberative functionalities. In this article, we present a temporal logic-based task planning and execution monitoring framework and its integration into a fully deployed rotor-based unmanned aircraft system developed in our laboratory. We use a very challenging emergency services application involving body identification and supply delivery as a vehicle for showing the potential use of such a framework in real-world applications. TALplanner, a temporal logic-based task planner, is used to generate mission plans. Building further on the use of TAL (Temporal Action Logic), we show how knowledge gathered from the appropriate sensors during plan execution can be used to create state structures, incrementally building a partial logical model representing the actual development of the system and its environment over time. We then show how formulas in the same logic can be used to specify the desired behavior of the system and its environment and how violations of such formulas can be detected in a timely manner in an execution monitor subsystem. The pervasive use of logic throughout the higher level deliberative layers of the system architecture provides a solid shared declarative semantics that facilitates the transfer of knowledge between different modules.

[1]  Doron Drusinsky Monitoring Temporal Rules Combined with Time Series , 2003, CAV.

[2]  P. Rudol,et al.  Human Body Detection and Geolocalization for UAV Search and Rescue Missions Using Color and Thermal Imagery , 2008, 2008 IEEE Aerospace Conference.

[3]  Sam Steel,et al.  Integrating Planning, Execution and Monitoring , 1988, AAAI.

[4]  Grigore Rosu,et al.  Monitoring Algorithms for Metric Temporal Logic Specifications , 2004, RV@ETAPS.

[5]  Koushik Sen,et al.  Program monitoring with LTL in EAGLE , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[6]  M. Sherman,et al.  A Preliminary Report , 1953 .

[7]  Howard Barringer,et al.  Rule Systems for Run-time Monitoring: from Eagle to RuleR , 2010, J. Log. Comput..

[8]  Jonas Kvarnström,et al.  Applying Domain Analysis Techniques for Domain-Dependent Control in TALplanner , 2002, AIPS.

[9]  E. Allen Emerson,et al.  Temporal and Modal Logic , 1991, Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics.

[10]  Patrick Doherty,et al.  Iterative Bounding LAO , 2010, ECAI.

[11]  Ola Pettersson,et al.  Execution monitoring in robotics: A survey , 2005, Robotics Auton. Syst..

[12]  Félix Ingrand,et al.  Interleaving Temporal Planning and Execution in Robotics Domains , 2004, AAAI.

[13]  Janos Gertler,et al.  Fault detection and diagnosis in engineering systems , 1998 .

[14]  Patrick Doherty,et al.  From images to traffic behavior - A UAV tracking and monitoring application , 2007, 2007 10th International Conference on Information Fusion.

[15]  Lars Karlsson,et al.  Reasoning about Concurrent Interaction , 1999, J. Log. Comput..

[16]  P. Mantegazza,et al.  RTAI: Real Time Application Interface , 2000 .

[17]  Nicolas Markey,et al.  Model Checking Restricted Sets of Timed Paths , 2004, CONCUR.

[18]  Frank van Harmelen,et al.  Handbook of Knowledge Representation , 2008, Handbook of Knowledge Representation.

[19]  Alessandro Saffiotti,et al.  An introduction to the anchoring problem , 2003, Robotics Auton. Syst..

[20]  Karen L. Myers CPEF: A Continuous Planning and Execution Framework , 1999, AI Mag..

[21]  Jonas Kvarnström,et al.  Elaboration tolerance through object-orientation , 2004, Artif. Intell..

[22]  Jonas Kvarnström,et al.  TALplanner and Other Extensions to Temporal Action Logic , 2005 .

[23]  Wolfgang Thomas,et al.  Handbook of Theoretical Computer Science, Volume B: Formal Models and Semantics , 1990 .

[24]  B. Faverjon,et al.  Probabilistic Roadmaps for Path Planning in High-Dimensional Con(cid:12)guration Spaces , 1996 .

[25]  Jan van Leeuwen,et al.  Handbook of Theoretical Computer Science, Vol. B: Formal Models and Semantics , 1994 .

[26]  Simone Duranti,et al.  Autonomous Landing of an Unmanned Helicopter based on Vision and Inertial Sensing , 2004, ISER.

[27]  Patrick Doherty,et al.  TALplanner: A Temporal Logic-Based Planner , 2001, AI Mag..

[28]  Patrick Doherty,et al.  A framework for reconfigurable path planning for autonomous unmanned aerial vehicles. , 2007 .

[29]  Patrick Doherty,et al.  Reasoning about Action and Change Using Occlusion , 1994, ECAI.

[30]  Froduald Kabanza,et al.  Reasoning about Robot Actions: A Model Checking Approach , 2001, Advances in Plan-Based Control of Robotic Agents.

[31]  Thomas A. Henzinger,et al.  Back to the future: towards a theory of timed regular languages , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[32]  Howard Barringer,et al.  Rule Systems for Run-Time Monitoring: From Eagleto RuleR , 2007, RV.

[33]  Per Olof Pettersson,et al.  Sampling-based Path Planning for an Autonomous Helicopter , 2006 .

[34]  David E. Wilkins,et al.  Interactive Execution Monitoring of Agent Teams , 2003, J. Artif. Intell. Res..

[35]  Steven M. LaValle,et al.  RRT-connect: An efficient approach to single-query path planning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[36]  Patrick Doherty,et al.  Temporal Action Logics , 2008, Handbook of Knowledge Representation.

[37]  Patrick Doherty,et al.  A knowledge processing middleware framework and its relation to the JDL data fusion model , 2005, 2005 7th International Conference on Information Fusion.

[38]  Richard W. Weyhrauch,et al.  Prolegomena to a Theory of Mechanized Formal Reasoning , 1980, Artif. Intell..

[39]  Rob Sherwood,et al.  Using Iterative Repair to Improve the Responsiveness of Planning and Scheduling , 2000, AIPS.

[40]  Marcus Bjäreland Model-based execution monitoring , 2001 .

[41]  Patrick Doherty,et al.  TAL: Temporal Action Logics Language Specification and Tutorial , 1998, Electron. Trans. Artif. Intell..

[42]  Fahiem Bacchus,et al.  Planning for temporally extended goals , 1996, Annals of Mathematics and Artificial Intelligence.

[43]  Giuseppe De Giacomo,et al.  Execution Monitoring of High-Level Robot Programs , 1998, KR.

[44]  Mariusz Wzorek,et al.  LinkMAV, a protoype rotary wing micro aerial vehicle , 2007 .

[45]  Grigore Rosu,et al.  Rewriting-Based Techniques for Runtime Verification , 2005, Automated Software Engineering.

[46]  Malik Ghallab,et al.  On Chronicles: Representation, On-line Recognition and Learning , 1996, KR.

[47]  Mariusz Wzorek,et al.  Control System Framework for Autonomous Robots Based on Extended State Machines , 2006, International Conference on Autonomic and Autonomous Systems (ICAS'06).

[48]  Patrick Doherty,et al.  Towards a Delegation Framework for Aerial Robotic Mission Scenarios , 2007, CIA.

[49]  Patrick Doherty,et al.  TALplanner: A temporal logic based forward chaining planner , 2001, Annals of Mathematics and Artificial Intelligence.

[50]  Patrick Doherty,et al.  Knowledge Representation and Unmanned Aerial Vehicles , 2005, Web Intelligence.

[51]  Patrick Doherty,et al.  Reconfigurable Path Planning for an Autonomous Unmanned Aerial Vehicle , 2006, 2006 International Conference on Hybrid Information Technology.

[52]  Bernd Finkbeiner,et al.  Checking Finite Traces Using Alternating Automata , 2004, Formal Methods Syst. Des..

[53]  Krzysztof Kuchcinski,et al.  The WITAS Unmanned Aerial Vehicle Project , 2000, ECAI.

[54]  Patrick Doherty,et al.  Managing Dynamic Object Structures using Hypothesis Generation and Validation , 2004 .

[55]  Patrick Doherty,et al.  DyKnow: An approach to middleware for knowledge processing , 2004, J. Intell. Fuzzy Syst..

[56]  Patrick Doherty,et al.  Computing Circumscription Revisited: A Reduction Algorithm , 1997, Journal of Automated Reasoning.

[57]  Patrik Haslum,et al.  A Distributed Architecture for Autonomous Unmanned Aerial Vehicle Experimentation , 2004, DARS.

[58]  Torsten Merz Building a System for Autonomous Aerial Robotics Research , 2004 .

[59]  Reid G. Simmons,et al.  Robust execution monitoring for navigation plans , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[60]  Erann Gat,et al.  Path planning and execution monitoring for a planetary rover , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[61]  Philippe Schnoebelen,et al.  Model Checking a Path , 2003, CONCUR.

[62]  David Harel,et al.  Statecharts: A Visual Formalism for Complex Systems , 1987, Sci. Comput. Program..

[63]  Patrick Doherty Advanced Research with Autonomous Unmanned Aerial Vehicles , 2004, KR.

[64]  Thomas A. Henzinger,et al.  The benefits of relaxing punctuality , 1991, PODC '91.

[65]  Richard Fikes,et al.  Monitored Execution of Robot Plans Producted by STRIPS , 1971, IFIP Congress.

[66]  G. Conte,et al.  Micro Unmanned Aerial Vehicle Visual Servoing for Cooperative Indoor Exploration , 2008, 2008 IEEE Aerospace Conference.

[67]  Patrick Doherty,et al.  TALplanner: an empirical investigation of a temporal logic-based forward chaining planner , 1999, Proceedings. Sixth International Workshop on Temporal Representation and Reasoning. TIME-99.

[68]  Stephan Merz,et al.  Model Checking , 2000 .

[69]  Reid G. Simmons,et al.  A task description language for robot control , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[70]  Mariusz Wzorek,et al.  From Motion Planning to Control - A Navigation Framework for an Autonomous Unmanned Aerial Vehicle , 2006 .

[71]  Patrick Doherty,et al.  Computing Circumscription Revisited: Preliminary Report , 1995, IJCAI.

[72]  Karen Zita Haigh,et al.  Planning, Execution and Learning in a Robotic Agent , 1998, AIPS.

[73]  Keith Golden,et al.  Plan Execution, Monitoring, and Adaptation for Planetary Rovers , 2000, Electron. Trans. Artif. Intell..

[74]  Patrick Doherty,et al.  A UAV Search and Rescue Scenario with Human Body Detection and Geolocalization , 2007, Australian Conference on Artificial Intelligence.

[75]  Michael Thielscher,et al.  Intelligent Execution Monitoring in Dynamic Environments , 2003, Fundam. Informaticae.

[76]  Manolis Koubarakis,et al.  Complexity Results for First-Order Theories of Temporal Constraints , 1994, KR.