HBP: an optimization technique to shorten the control cycle time of the Neural Network Controller that provides dynamic buffer tuning to eliminate overflow at user level

The NNC (Neural Network Controller) automatically tunes the buffer SIze at the user/server level to eliminate any chance of overflow in the client/server interaction over a TCP logical channel. Together with the buffer tuning operations at the system/router level (e.g. the AQM (Active Queue Management) activities) they form a unified solution. The power and stability of the NNC was verified over the Internet, but the result shows that the drawback of the NNC is its long control cycle time. This drawback hinders the deployment of the NNC in the real-time applications. To overcome this we propose the novel HBP (Hessian Based Pruning) optimization technique. This technique operates as a renewal process, and within the service life of the Optimized NNC (O-NNC) the optimization operation repeats as renewal cycles. The feed-forward neural network configuration of the O-NNe is optimized in every cycle that involves two phases. In its original un-optimized form the NNC runs as a twin system of two modules:" Chief + Learner". The O-NNC always starts with the un-optimized configuration. In the first phase the weights for the Learner's neural network arcs are computed and sorted. Those arcs with weights insignificant to the control convergence speed and precision are marked. The marking is based on "dynamic sensitivity analysis" that utilizes the HBP technique. In the second phase the Chief optimizes the neural network by excluding/skipping the marked arcs. The aim is to shorten the computation for the control cycle. The "HBP+NNC' is the basis of the O-NNC model, which essentially uses virtual pruning because the marked arcs are excluded from the computation but not physically removed. While the Chief is carrying out actual dynamic buffer tuning the Learner undergoes training. The O-NNC model is verified by running the Java-based prototype on the Aglets mobile agent platform in the Internet environment. The results are positive and indicate that the HBP technique indeed yields a shorter O-NNC control cycle time than the original un-optimized NNC in a consistently manner.

[1]  Tharam S. Dillon,et al.  A fault-tolerant data communication setup to improve reliability and performance for Internet based distributed applications , 1999, Proceedings 1999 Pacific Rim International Symposium on Dependable Computing.

[2]  James Aweya,et al.  Neurocontroller for buffer overload control in a packet switch , 1998 .

[3]  James Aweya,et al.  Multi-level active queue management with dynamic thresholds , 2002, Comput. Commun..

[4]  Sally Floyd,et al.  Wide area traffic: the failure of Poisson modeling , 1995, TNET.

[5]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[6]  Isi Mitrani,et al.  Probabilistic Modelling , 1998 .

[7]  Tharam S. Dillon,et al.  M/sup 3/RT: an Internet end-to-end performance measurement approach for real-time applications with mobile agents , 2002, Proceedings International Symposium on Parallel Architectures, Algorithms and Networks. I-SPAN'02.

[8]  Panos J. Antsaklis,et al.  The dependence identification neural network construction algorithm , 1996, IEEE Trans. Neural Networks.

[9]  Deborah Estrin,et al.  Recommendations on Queue Management and Congestion Avoidance in the Internet , 1998, RFC.

[10]  Allan K. Y. Wong,et al.  A novel adaptive fuzzy logic controller (FLC) to improve Internet channel reliability and response timeliness , 2003, Proceedings of the Eighth IEEE Symposium on Computers and Communications. ISCC 2003.

[11]  Halbert White,et al.  On learning the derivatives of an unknown mapping with multilayer feedforward networks , 1992, Neural Networks.

[12]  Sang-Hoon Oh,et al.  Adaptive learning rate and limited error signal for multilayer perceptrons with n-th order cross-entropy error , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[13]  Allan K. Y. Wong,et al.  Genetic Algorithm and PID Control Together for Dynamic Anticipative Marginal Buffer Management: An Effective Approach to Enhance Dependability and Performance for Distributed Mobile Object-Based Real-Time Computing over the Internet , 2002, J. Parallel Distributed Comput..

[14]  Ibrahim Matta,et al.  On the origin of power laws in Internet topologies , 2000, CCRV.

[15]  Eng-Chong Tan,et al.  Pruning neural networks during training by backpropagation , 1994, Proceedings of TENCON'94 - 1994 IEEE Region 10's 9th Annual International Conference on: 'Frontiers of Computer Technology'.

[16]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[17]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[18]  Fengyuan Ren,et al.  Design of a fuzzy controller for active queue management , 2002, Computer Communications.

[19]  Martin T. Hagan,et al.  Neural network design , 1995 .

[20]  Van Jacobson,et al.  Random early detection gateways for congestion avoidance , 1993, TNET.

[21]  Martin May,et al.  Analytic evaluation of RED performance , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[22]  Victor Firoiu,et al.  A study of active queue management for congestion control , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[23]  T. V. Lakshman,et al.  The performance of TCP/IP for networks with high bandwidth-delay products and random loss , 1997, TNET.

[24]  Shan Xiuming,et al.  Design of a fuzzy controller for active queue management , 2002 .

[25]  Tharam S. Dillon,et al.  An adaptive buffer management algorithm for enhancing dependability and performance in mobile-object-based real-time computing , 2001, Fourth IEEE International Symposium on Object-Oriented Real-Time Distributed Computing. ISORC 2001.

[26]  Robert Tappan Morris,et al.  Dynamics of random early detection , 1997, SIGCOMM '97.

[27]  Tharam S. Dillon,et al.  HBM: A Suitable Neural Network Pruning Technique to Optimize the Execution Time of the Novel Neural Network Controller (NNC) that Eliminates Buffer Overflow , 2003, PDPTA.