Multi-level operational C2 architecture modeling via hierarchically structured semi-Markov decision processes

This paper presents a multi-level operational command and control (C2) architecture that is applicable to the Navy's maritime operations centers (MOCs) for assessing, planning and executing multiple missions and tasks across a range of military operations. The control architecture consists of three levels: strategic level control (SLC), operational level control (OLC) and tactical level control (TLC). In addition to coordination within each level, two specific coordination layers are identified at the strategic-operational level control (SLC-OLC) and at the operational-tactical level control (OLC-TLC) interfaces. We employ a semi-Markov decision process (SMDP) approach at the SLC-OLC interface layer to optimize DIME (diplomatic, information, military and economic) actions based on national resource priorities. A distributed SMDP approach is applied to action-goal attainment (AGA) graphs at the OLC-TLC interface layer to address the mission monitoring/planning issues and to optimize the courses of action based on the outcomes of asset-task allocation at the TLC. The optimization of a hierarchy of SMDPs is accomplished by exchanging performance and mission priority information through the interface layers. Specifically, the times between decision epochs at the SLC-OLC layer are determined by the mission completion times at the OLC-TLC layer, while the mission priorities (weights) to be used in mission planning at the OLC-TLC layer are determined by DIME actions at the SLC-OLC layer. In short, we model the hierarchical decision making process by exchanging the outcomes of SMDPs between the SLC-OLC and the OLC-TLC layers to find the best courses of actions for a range of military operations.

[1]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[2]  David L. Kleinman,et al.  An Investigation of ISR Coordination and Information Presentation Strategies to Support Expeditionary Strike Groups , 2007 .

[3]  Krishna R. Pattipati,et al.  Multi-level operational C2 holonic reference architecture modeling for MHQ with MOC , 2009 .

[4]  Krishna R. Pattipati,et al.  A Markov Decision Problem Approach to Goal Attainment , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[5]  Ronald E. Parr,et al.  Hierarchical control and learning for markov decision processes , 1998 .

[6]  Arnaud Doucet,et al.  A policy gradient method for SMDPs with application to call admission control , 2002, 7th International Conference on Control, Automation, Robotics and Vision, 2002. ICARCV 2002..

[7]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[8]  Leslie Pack Kaelbling,et al.  Approximate Planning in POMDPs with Macro-Actions , 2003, NIPS.

[9]  U. Pape,et al.  Implementation and efficiency of Moore-algorithms for the shortest route problem , 1974, Math. Program..

[10]  Richard Bellman,et al.  ON A ROUTING PROBLEM , 1958 .

[11]  Krishna R. Pattipati,et al.  Integration of a Holonic Organizational Control Architecture and Multiobjective Evolutionary Algorithm for Flexible Distributed Scheduling , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[12]  Sridhar Mahadevan,et al.  Decision-Theoretic Planning with Concurrent Temporally Extended Actions , 2001, UAI.