A Multiagent Dynamic Assessment Approach for Water Quality Based on Improved Q-Learning Algorithm

The dynamic water quality assessment is a challenging and critical issue in water resource management systems. To deal with this complex problem, a dynamic water assessment model based on multiagent technology is proposed, and an improved Q-learning algorithm is used in this paper. In the proposed Q-learning algorithm, a fuzzy membership function and a punishment mechanism are introduced to improve the learning speed of Q-learning algorithm. The dynamic water quality assessment for different regions and the prewarning of water pollution are achieved by using an interaction factor in the proposed approach. The proposed approach can deal with various situations, such as static and dynamic water quality assessment. The experimental results show that the water quality assessment based on the proposed approach is more accurate and efficient than the general methods.

[1]  Xu Zu-xin,et al.  Comprehensive Water Quality Identification Index for EnvironmentalQuality Assessment of Surface Water , 2005 .

[2]  Liu Kun.  Advection-diffusion models establishment of water-pollution accident in middle and lower reaches of Hanjiang river , 2008 .

[3]  Wen Sun,et al.  City Optimal Allocation of Water Resources Research Based on Sustainable Development , 2012 .

[4]  Wenwu Yu,et al.  An Overview of Recent Progress in the Study of Distributed Multi-Agent Coordination , 2012, IEEE Transactions on Industrial Informatics.

[5]  Simon X. Yang,et al.  Hierarchical Approximate Policy Iteration With Binary-Tree State Space Decomposition , 2011, IEEE Transactions on Neural Networks.

[6]  Zhao Xiao-hu,et al.  Research and Application of Reinforcement Learning Based on Constraint MDP in Coal Mine , 2009 .

[7]  Wen-bo Xu,et al.  Fuzzy Q-learning in continuous state and action space , 2010 .

[8]  Guanghui Wen,et al.  Consensus of multi‐agent systems with nonlinear dynamics and sampled‐data information: a delayed‐input approach , 2013 .

[9]  Feng Liu,et al.  Fuzzy optimization of water resources project scheme based on improved grey relation analysis , 2011, 2011 3rd International Conference on Computer Research and Development.

[10]  Zhihong Zou,et al.  Quantificational analysis on progress of river water quality in China. , 2009, Journal of environmental sciences.

[11]  Majid Nili Ahmadabadi,et al.  Exploration and exploitation balance management in fuzzy reinforcement learning , 2010, Fuzzy Sets Syst..

[12]  Ruchuan Wang,et al.  Secure coverage-preserving node scheduling scheme using energy prediction for wireless sensor networks , 2010 .

[13]  Hiroshi Matsuo,et al.  Multiagent reinforcement learning with the partly high-dimensional state space , 2006, Systems and Computers in Japan.

[14]  Andrea Bonarini,et al.  Reinforcement distribution in fuzzy Q-learning , 2009, Fuzzy Sets Syst..

[15]  S. Rodgher,et al.  Evaluation of surface water quality in aquatic bodies under the influence of uranium mining (MG, Brazil) , 2013, Environmental Monitoring and Assessment.

[16]  Qing-qing Wang,et al.  Research and Application of Reinforcement Learning Based on Constraint MDP in Coal Mine , 2009, 2009 WRI World Congress on Computer Science and Information Engineering.

[17]  Hiroshi Aoki,et al.  Development of a new lightning detection system , 2006 .

[18]  Chenming Li,et al.  Solution Method Of Optimal Scheme Set For Water Resources Scheduling Group Decision-Mahing Based On Multi-Agent Computation , 2011, Intell. Autom. Soft Comput..

[19]  Li Ren,et al.  An Intelligent Decision Support System of Lake Water Pollution Control Based on Multi-agent Model , 2009, 2009 International Conference on Computational Intelligence and Security.

[20]  Michael L. Littman,et al.  Value-function reinforcement learning in Markov games , 2001, Cognitive Systems Research.

[21]  Ni Shen.  Application of BP Neural Network Model in Groundwater Quality Evaluation , 2000 .

[22]  Florin Leon,et al.  Emergent Behaviors in Social Networks of Adaptive Agents , 2012 .

[23]  Qing Wu,et al.  Landscape river water quality assessment by Nemerow pollution index , 2010, 2010 International Conference on Mechanic Automation and Control Engineering.

[24]  Xiangfeng Luo,et al.  Research on cooperation and learning in multi-agent system , 2002, 6th International Conference on Signal Processing, 2002..

[25]  Pengcheng Zhang,et al.  A novel multi-agent reinforcement learning approach for job scheduling in Grid computing , 2011, Future Gener. Comput. Syst..

[26]  Jian Shu,et al.  A Water Quality Monitoring Method Based on Fuzzy Comprehensive Evaluation in Wireless Sensor Networks , 2012, J. Networks.

[27]  Fang Huang,et al.  Spatio-temporal patterns and source apportionment of pollution in Qiantang River (China) using neural-based modeling and multivariate statistical techniques , 2011 .

[28]  N. Pochai.  A Numerical Treatment of Nondimensional Form of Water Quality Model in a Nonuniform Flow Stream Using Saulyev Scheme , 2011 .

[29]  Jing Wang,et al.  An application of agent-based simulation to knowledge sharing , 2009, Decis. Support Syst..

[30]  Xin Xu,et al.  Kernel-Based Least Squares Policy Iteration for Reinforcement Learning , 2007, IEEE Transactions on Neural Networks.

[31]  Z. Q. Zhao,et al.  Real Time Monitoring of Surface Water Pollution using Microwave System , 2008 .