Solving Stochastic Nonlinear Resource Allocation Problems Using a Hierarchy of Twofold Resource Allocation Automata

In a multitude of real-world situations, resources must be allocated based on incomplete and noisy information. However, in many cases, incomplete and noisy information render traditional resource allocation techniques ineffective. The decentralized Learning Automata Knapsack Game (LAKG) was recently proposed for solving one such class of problems, namely the class of Stochastic Nonlinear Fractional Knapsack Problems. Empirically, the LAKG was shown to yield a superior performance when compared to methods which are based on traditional parameter estimation schemes. This paper presents a completely new online Learning Automata (LA) system, namely the Hierarchy of Twofold Resource Allocation Automata (H-TRAA). In terms of contributions, we first of all, note that the primitive component of the H-TRAA is a Twofold Resource Allocation Automaton (TRAA) which possesses novelty in the field of LA. Second, the paper contains a formal analysis of the TRAA, including a rigorous proof for its convergence. Third, the paper proves the convergence of the H-TRAA itself. Finally, we demonstrate empirically that the H-TRAA provides orders of magnitude faster convergence compared to the LAKG for simulated data pertaining to two-material unit-value functions. Indeed, in contrast to the LAKG, the H-TRAA scales sublinearly. Consequently, we believe that the H-TRAA opens avenues for handling demanding real-world applications such as the allocation of sampling resources in large-scale Web accessibility assessment problems. We are currently working on applying the H-TRAA solution to the web-polling and sample-size detection problems applicable to the world wide web.

[1]  E. Steinberg,et al.  A Preference Order Dynamic Program for a Knapsack Problem with Stochastic Rewards , 1979 .

[2]  David D. Yao,et al.  The Stochastic Knapsack Revisited: Switch-Over Policies and Dynamic Pricing , 2008, Oper. Res..

[3]  Michael J. Fry,et al.  An agent-based stochastic ruler approach for a stochastic knapsack problem with sequential competition , 2010, Comput. Oper. Res..

[4]  David D. Yao,et al.  The Stochastic Knapsack Revisited: Structure, Switch-Over Policies, and Dynamic Pricing , 2002 .

[5]  B. John Oommen,et al.  Stochastic searching on the line and its applications to parameter learning in nonlinear optimization , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[6]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[7]  Keith W. Ross,et al.  The stochastic knapsack problem , 1989, IEEE Trans. Commun..

[8]  B. John Oommen,et al.  Absorbing and Ergodic Discretized Two-Action Learning Automata , 1986, IEEE Trans. Syst. Man Cybern..

[9]  J. Vondrák,et al.  Approximating the Stochastic Knapsack Problem: The Benefit of Adaptivity , 2008 .

[10]  Sandeep Pandey,et al.  Monitoring the dynamic web to respond to continuous queries , 2003, WWW '03.

[11]  M. Thathachar,et al.  Networks of Learning Automata: Techniques for Online Stochastic Optimization , 2003 .

[12]  Toshihide Ibaraki,et al.  Fractional knapsack problems , 1977, Math. Program..

[13]  M. L. Tsetlin,et al.  Automaton theory and modeling of biological systems , 1973 .

[14]  Philip S. Yu,et al.  Optimal crawling strategies for web search engines , 2002, WWW '02.

[15]  M. A. L. Thathachar,et al.  Networks of Learning Automata , 2004 .

[16]  B. John Oommen,et al.  On Allocating Limited Sampling Resources Using a Learning Automata-based Solution to the Fractional Knapsack Problem , 2006, Intelligent Information Systems.

[17]  B. John Oommen,et al.  Learning Automata-Based Solutions to the Nonlinear Fractional Knapsack Problem With Applications to Optimal Resource Allocation , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  G. K. Bhattacharyya,et al.  Statistical Concepts And Methods , 1978 .

[19]  B. John Oommen,et al.  Stochastic learning-based weak estimation of multinomial random variables and its applications to pattern recognition in non-stationary environments , 2006, Pattern Recognit..

[20]  Bennett Fox,et al.  Discrete Optimization Via Marginal Analysis , 1966 .

[21]  Bala Shetty,et al.  The nonlinear knapsack problem - algorithms and applications , 2002, Eur. J. Oper. Res..

[22]  Joseph C. Hartman,et al.  An approximate dynamic programming approach to solving a dynamic, stochastic multiple knapsack problem , 2009, Int. Trans. Oper. Res..

[23]  B. John Oommen,et al.  A Hierarchy of Twofold Resource Allocation Automata Supporting Optimal Web Polling , 2008, IEA/AIE.

[24]  J. Sachs.  A stochastic knapsack model for the capacity evaluation of (multi-) radio access networks , 2006 .

[25]  B. John Oommen,et al.  A Hierarchy of Twofold Resource Allocation Automata Supporting Optimal Sampling , 2009, IEA/AIE.

[26]  B.J. Oommen,et al.  Determining Optimal Polling Frequency Using a Learning Automata-based Solution to the Fractional Knapsack Problem , 2006, 2006 IEEE Conference on Cybernetics and Intelligent Systems.