Metric assessment of autonomous capabilities in unmanned maritime vehicles

Autonomous capabilities in Unmanned Maritime Vehicles (UMVs) have been pushing the technology beyond imaginable limits in order to deal with more and more complex ocean and sea missions. However, there is still no agreement on the aspects that should be evaluated to know whether a UMV satisfies the degree or level of autonomy expected. This paper reviews current approaches to assess autonomous behavior in UMVs. It also recaps the different development and operation aspects taken into account by researchers and practitioners to evaluate autonomy in unmanned systems from diverse domains. This provides a baseline to study methodologies to assess maritime autonomy so that metrics and assessment criteria can be defined for UMVs. This paper also presents statistical results from a comparison of existing assessment frameworks for autonomy in different domains. Outcomes from this study are critical to devise guidelines for metric autonomy assessment in UMVs. In addition, a discussion on relevant aspects of a autonomy assessment process for UMVs, and future research directions are presented.

[1]  Yavnai Rafael,et al.  An Information-Based Approach for System Autonomy Metrics Part I : Metrics Definition Arie , 2003 .

[2]  Natasha McCarthy,et al.  Autonomous Systems: Social, Legal and Ethical Issues , 2009 .

[3]  Adam Jacoff,et al.  Performance measures framework for unmanned systems (PerMFUS): initial perspective , 2009, PerMIS.

[4]  Lora G. Weiss,et al.  Intelligent Autonomy and Performance Measures for Coordinated Unmanned Vehicles , 2004 .

[5]  Jack Ring,et al.  Metrics for co-evolving autonomous systems , 2009, PerMIS.

[6]  Ryan W. Proud,et al.  Methods for Determining the Level of Autonomy to Design into a Human Spaceflight Vehicle: A Function Specific Approach , 2003 .

[7]  J. Gunderson,et al.  Intelligence ≠ Autonomy ≠ Capability , 2004 .

[8]  M. Steinberg,et al.  Metrics for Intelligent Autonomy , 2004 .

[9]  Christopher S. Martin.  Agent Autonomy: Specification, Measurement, and Dynamic Adjustment , 1999 .

[10]  Larry A. Young.  Feasibility of Turing-Style Tests for Autonomous Aerial Vehicle "Intelligence" , 2007 .

[11]  Yibo Li,et al.  Hierarchical Model and Evaluation Method for Autonomy Levels of Unmanned Platforms , 2012 .

[12]  Kirk L. Kroeker.  Living machines , 2008, CACM.

[13]  Jan D. Gehrke.  Evaluating situation awareness of autonomous systems , 2008, PerMIS.

[14]  James S. Albus,et al.  Specifying autonomy levels for unmanned systems: interim report , 2004, SPIE Defense + Commercial Sensing.

[15]  Scott E. McIntosh.  The Wingman-Philosopher of MiG Alley: John Boyd and the OODA Loop , 2011 .

[16]  Olivier Boissier,et al.  Autonomy in Multi-agent Systems: A Classification Attempt , 2003, Agents and Computational Autonomy.

[17]  James G. Bellingham,et al.  Performance metrics for oceanographic surveys with autonomous underwater vehicles , 2001 .

[18]  Michael R. Benjamin,et al.  Autonomous Underwater Vehicles: Trends and Transformations , 2005 .

[19]  Steve Scheding,et al.  Developments and Challenges for Autonomous Unmanned Vehicles - A Compendium , 2010, Intelligent Systems Reference Library.

[20]  James S. Albus,et al.  Autonomy levels for unmanned systems (ALFUS) framework: an update , 2005, SPIE Defense + Commercial Sensing.

[21]  Bruce T Clough,et al.  Metrics, Schmetrics! How The Heck Do You Determine A UAV's Autonomy Anyway , 2002 .