Multitarget tracking via restless bandit marginal productivity indices and Kalman filter in discrete time

This paper designs, evaluates, and tests a tractable priority-index policy for scheduling target updates in a discrete-time multitarget tracking model, which aims to be close to optimal relative to a discounted or average performance objective accounting for tracking-error variance and measurement costs. The policy is to be used by M phased-array radars who coordinate to track the positions of N targets moving according to independent scalar Gauss-Markov linear dynamics, which allows use of the Kalman filter for track estimation. The paper exploits the natural problem formulation as a multiarmed restless bandit problem (MARBP) with real-state projects subject to deterministic dynamics by deploying Whittle's (1988) index policy for the MARBP. The challenging issues of indexability (existence of the index) and index evaluation are resolved by applying a method recently introduced by the first author for the analysis of real-state restless bandits. Preliminary computational results are reported demonstrating the tractability of index evaluation and comparing the MP index policy against myopic policies advocated in previous work.

[1]  J. Gittins.  Bandit processes and dynamic allocation indices , 1979 .

[2]  P. Whittle.  Restless bandits: activity allocation in a changing world , 1988, Journal of Applied Probability.

[3]  Barbara F. La Scala,et al.  Optimal target tracking with restless bandits , 2006, Digit. Signal Process..

[4]  J. Nio-Mora.  An Index Policy for Dynamic Fading-Channel Allocation to Heterogeneous Mobile Users with Partial Observations , 2008, 2008 Next Generation Internet Networks.

[5]  R. Weber,et al.  ON AN INDEX POLICY FOR RESTLESS BANDITS , 1990 .

[6]  José Niño-Mora,et al.  Dynamic priority allocation via restless bandit marginal productivity indices , 2007, 2304.06115.

[7]  José Niño Mora.  Restless Bandits, Partial Conservation Laws and Indexability , 2000 .

[8]  José Niño-Mora,et al.  Dynamic allocation indices for restless projects and queueing admission control: a polyhedral approach , 2002, Math. Program..

[9]  J. Nino-Mora.  Marginal productivity index policies for scheduling multiclass wireless transmissions , 2006, 2006 2nd Conference on Next Generation Internet Design and Engineering, 2006. NGI '06..

[10]  G. V. Keuk,et al.  On phased-array radar tracking and parameter control , 1993 .

[11]  D. Stromberg.  Scheduling of track updates in phased array radars , 1996, Proceedings of the 1996 IEEE National Radar Conference.

[12]  O. Hernández-Lerma,et al.  Further topics on discrete-time Markov control processes , 1999 .

[13]  José Niño-Mora,et al.  Restless Bandit Marginal Productivity Indices, Diminishing Returns, and Optimal Control of Make-to-Order/Make-to-Stock M/G/1 Queues , 2006, Math. Oper. Res..

[14]  Robin J. Evans,et al.  Hidden Markov model multiarm bandits: a methodology for beam scheduling in multitarget tracking , 2001, IEEE Trans. Signal Process..

[15]  P. Whittle.  Restless Bandits: Activity Allocation in a Changing World , 1988 .

[16]  Young-Hun Jung,et al.  Optimal scheduling of track updates in phased array radars , 1998 .

[17]  William Moran,et al.  Application of Sensor Scheduling Concepts to Radar , 2008 .

[18]  José Niño-Mora,et al.  Marginal productivity index policies for scheduling a multiclass delay-/loss-sensitive queue , 2006, Queueing Syst. Theory Appl..

[19]  EWRD,et al.  Optimal Policy for Scheduling of Gauss-Markov Systems , 2004 .

[20]  José Niño-Mora.  A Restless Bandit Marginal Productivity Index for Opportunistic Spectrum Access with Sensing Errors , 2009, NET-COOP.

[21]  J. Niño-Mora.  RESTLESS BANDITS, PARTIAL CONSERVATION LAWS AND INDEXABILITY , 2001 .

[22]  J. Nio-Mora.  Restless Bandit Marginal Productivity Indices, Diminishing Returns, and Optimal Control of Make-to-Order/Make-to-Stock M/G/1 Queues , 2006 .