A generalized reinforcement learning scheme for random neural networks

We propose a reinforcement learning scheme for random neural networks (RNN) that update network weights based on the relative reward obtained by past actions. This new learning scheme achieves improved performance in random environments under certain conditions, in particular, when the number of actions is small. We also explore hybrid learning structures, where a RNN functions as a meta decision unit that learns how to best delegate the action decision to a second layer of decision units. An extensive simulation study demonstrates the performance of these learning methods applied to minimizing response times in a distributed task assignment scenario. Results are compared to the cognitive packet networks’ reinforcement learning scheme, a reward-only variation, and several non-learning methods. The study reveals that no single method can produce the best performance results in all cases. However, both the generalized reinforcement learning scheme and the application of meta decision units can help to achieve improved performance in specific cases.

[1]  Gerardo Rubino,et al.  A GRASP algorithm with RNN based local search for designing a WAN access network , 2004, Electron. Notes Discret. Math..

[2]  Gerardo Rubino,et al.  Levenberg - Marquardt Training Algorithms for Random Neural Networks , 2011, Comput. J..

[3]  Erol Gelenbe,et al.  The Random Neural Network Model for Texture Generation , 1992, Int. J. Pattern Recognit. Artif. Intell..

[4]  Erol Gelenbe,et al.  Image and video compression , 1998 .

[5]  Erol Gelenbe,et al.  Design and performance of cognitive packet networks , 2001, Perform. Evaluation.

[6]  R. Lent Linear QoS Goals of Additive and Concave Metrics in Ad Hoc Cognitive Packet Routing , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[8]  Gerardo Rubino,et al.  A more powerful random neural network model in supervised learning applications , 2013, 2013 International Conference on Soft Computing and Pattern Recognition (SoCPaR).

[9]  Erol Gelenbe,et al.  Parallel Algorithm for Colour Texture Generation Using the Random Neural Network Model , 1992, Int. J. Pattern Recognit. Artif. Intell..

[10]  Erol Gelenbe,et al.  Routing Emergency Evacuees with Cognitive Packet Networks , 2013, ISCIS.

[11]  Erol Gelenbe,et al.  Minimum Graph Vertex Covering with the Random Neural Network , 1992, Computer Science and Operations Research.

[12]  Erol Gelenbe,et al.  Random Neural Network for Emergency Management , 2010 .

[13]  Erol Gelenbe,et al.  Detecting Denial of Service Attacks with Bayesian Classifiers and the Random Neural Network , 2007, 2007 IEEE International Fuzzy Systems Conference.

[14]  Erol Gelenbe,et al.  Video quality and traffic QoS in learning-based subsampled and receiver-interpolated video sequences , 2000, IEEE Journal on Selected Areas in Communications.

[15]  Andreas Stafylopatis,et al.  Pictorial Information Retrieval Using the Random Neural Network , 1992, IEEE Trans. Software Eng..

[16]  Will Serrano,et al.  An Intelligent Internet Search Assistant Based on the Random Neural Network , 2016, AIAI.

[17]  Jianjun Wu,et al.  Dynamical Random Neural Network Approach to a Problem of Optimal Resource Allocation , 2005, IWANN.

[18]  Erol Gelenbe,et al.  Image processing with the random neural network , 1998, Electronic Imaging.

[19]  Gerardo Rubino,et al.  A study of real-time packet video quality using random neural networks , 2002, IEEE Trans. Circuits Syst. Video Technol..

[20]  Erol Gelenbe,et al.  Self-aware networks and QoS , 2004, Proceedings of the IEEE.

[21]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[22]  Giuseppe Lo Re,et al.  The Random Neural Network Model for the On-Line Multicast Problem , 2004, WIRN.

[23]  Erol Gelenbe,et al.  Low bit-rate video compression with neural networks and temporal subsampling , 1996, Proc. IEEE.

[24]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[25]  Taskin Koçak,et al.  Learning in the feed-forward Random Neural Network: A Critical Review , 2010, ISCIS.

[26]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[27]  Erol Gelenbe,et al.  Random neural network texture model , 2000, Electronic Imaging.

[28]  Anoop Ghanwani,et al.  Improved Neural Heuristics for Multicast Routing , 1997, IEEE J. Sel. Areas Commun..

[29]  Taskin Koçak,et al.  Learning in the feed-forward random neural network: A critical review , 2011, Perform. Evaluation.

[30]  Erol Gelenbe,et al.  Learning in the Recurrent Random Neural Network , 1992, Neural Computation.

[31]  Georgios Loukas,et al.  A Denial of Service Detector based on Maximum Likelihood Detection and the Random Neural Network , 2007, Comput. J..

[32]  Alberto Prieto,et al.  Proceedings of the 8th international conference on Artificial Neural Networks: computational Intelligence and Bioinspired Systems , 2005 .

[33]  Erol Gelenbe,et al.  Random Neural Networks with Negative and Positive Signals and Product Form Solution , 1989, Neural Computation.

[34]  Georgios Loukas,et al.  A Biologically Inspired pired Denial of Service Detector Using the Random Neural Network , 2007, 2007 IEEE Internatonal Conference on Mobile Adhoc and Sensor Systems.

[35]  S. Russel and P. Norvig,et al.  “Artificial Intelligence – A Modern Approach”, Second Edition, Pearson Education, 2003. , 2015 .

[36]  Erol Gelenbe,et al.  Power-aware ad hoc cognitive packet networks , 2004, Ad Hoc Networks.

[37]  Ferhan Pekergin,et al.  Dynamical random neural network approach to the traveling salesman problem , 1993, Proceedings of IEEE Systems Man and Cybernetics Conference - SMC.

[38]  Kumpati S. Narendra,et al.  Learning Automata - A Survey , 1974, IEEE Trans. Syst. Man Cybern..

[39]  U Halici,et al.  Reinforcement learning in random neural networks for cascaded decisions. , 1997, Bio Systems.

[40]  Erol Gelenbe,et al.  Simulation with learning agents , 2001, Proc. IEEE.

[41]  Erol Gelenbe,et al.  Area-based results for mine detection , 2000, IEEE Trans. Geosci. Remote. Sens..

[42]  Antonello Rizzi,et al.  Extended Random Neural Networks , 2002, WIRN.

[43]  Erol Gelenbe,et al.  Minimum Graph Covering with the Random Neural Network Model , 1992 .

[44]  Huibo Bi,et al.  Multi-path routing in emergency with health-aware classification , 2015, 2015 2nd International Conference on Information and Communication Technologies for Disaster Management (ICT-DM).

[45]  Erol Gelenbe,et al.  Stability of the Random Neural Network Model , 1990, Neural Computation.

[46]  Erol Gelenbe,et al.  Neural networks for volumetric MR imaging of the brain , 1996, Proceedings of International Workshop on Neural Networks for Identification, Control, Robotics and Signal/Image Processing.

[47]  Yi Shen,et al.  Image Segmentation Based on Random Neural Network Model and Gabor Filters , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[48]  Pierre Tirilly,et al.  Evaluating Users' Satisfaction in Packet Networks Using Random Neural Networks , 2006, ICANN.

[49]  E. Gelenbe Une généralisation probabiliste du problème SAT , 1992 .

[50]  Gulay Oke,et al.  Likelihood ratios and recurrent random neural networks in detection of denial of service attacks , 2007 .

[51]  Erol Gelenbe,et al.  Video compression with random neural networks , 1996, Proceedings of International Workshop on Neural Networks for Identification, Control, Robotics and Signal/Image Processing.