Autonomic learning through stochastic games for rational allocation of scarce medical resources

The confront of medical ethics, 'rational allocation of scarce medical resources' practically appears as uncertainty in secondary-spectrum-usage in a cognitive Body Area Network(BAN). In this context, we have formulated RMSRA-VB(Rational Medical Scarce Resource Allocator Virtual Backbone) as an initial step to provide autonomy in wireless agents so that they can adapt to that uncertainty caused by collision-avoidance and EMI(Electro-Magnetic Interference)avoidance. Then, we have proposed a distributed autonomic learning framework, RMSRA-QL-POSG(RMSRA using Q-Learning in a Partial Observable Stochastic Game Model). RMSRAQL-POSG is the first Q-learing algorithm in a POSG Game Model. By RMSRA-QL-POSG, wireless agents learn to utlize secondary-channel in uncertain cognitive BAN. Our probabilistic analysis proves how RMSRA-QL-POSG is successful in inferring uncertainty , in terms of value/reward-over-belief calculation in time horizons, T=1 and T=2, by considering wireless agents' states(monitoring, emergency) and observations (EMI-effect and collision-avoidance).Proof of convergence and complexity analysis of RMSRA-QL-POSG are also presented.

[1]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[2]  R. Babuska,et al.  A COMPREHENSIVE SURVEY OF MULTIAGENT REINFORCEMENT , 2013 .

[3]  E. Emanuel,et al.  Principles for allocation of scarce medical interventions , 2009, The Lancet.

[4]  Jianfeng Wang,et al.  Emerging cognitive radio applications: A survey , 2011, IEEE Communications Magazine.

[5]  Shlomo Zilberstein,et al.  Formal models and algorithms for decentralized decision making under uncertainty , 2008, Autonomous Agents and Multi-Agent Systems.

[6]  Choong Seon Hong,et al.  A policy based management framework for machine to machine networks and services , 2011, 2011 13th Asia-Pacific Network Operations and Management Symposium.

[7]  Choong Seon Hong,et al.  Fault tolerant virtual backbone for minimum temperature in in vivo sensor networks , 2012, 2012 IEEE International Conference on Communications (ICC).