Reinforcement Learning Techniques for the Control of WasteWater Treatment Plants

Since water pollution is one of the most serious environmental problems today, control of wastewater treatment plants (WWTPs) is a crucial issue nowadays and stricter standards for the operation of WWTPs have been imposed by authorities. One of the main problems in the automation of the control of Wastewater Treatment Plants (WWTPs) appears when the control system does not respond as it should because of changes on influent load or flow. Thus, it is desirable the development of autonomous systems that learn from interaction with a WWTP and that can operate taking into account changing environmental circumstances. In this paper we present an intelligent agent using reinforcement learning for the oxygen control in the N-Ammonia removal process in the well known Benchmark Simulation Model no.1 (BSM1). The aim of the approach presented in this paper is to minimize the operation cost changing the set-points of the control system autonomously.

[1]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[2]  N Hvala,et al.  Comparison of control strategies for nitrogen removal in an activated sludge process in terms of operating costs: a simulation study. , 2007, Water research.

[3]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[4]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[5]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[6]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Vol. II , 1976 .

[7]  Miquel Sànchez-Marrè,et al.  Artificial Intelligence and Environmental Decision Support Systems , 2000, Applied Intelligence.

[8]  Juan M. Lema,et al.  Diagnosis of acidification states in an anaerobic wastewater treatment plant using a fuzzy-based expert system , 2004 .

[9]  Bengt Carlsson,et al.  Cost-efficient operation of a denitrifying activated sludge process. , 2007, Water research.

[10]  Yueh-Min Huang,et al.  A blog article recommendation generating mechanism using an SBACPSO algorithm , 2009, Expert Syst. Appl..

[11]  Shu-Hsien Liao,et al.  Expert system methodologies and applications - a decade review from 1995 to 2004 , 2005, Expert Syst. Appl..

[12]  Yan Wan,et al.  A fast predicting neural fuzzy model for on-line estimation of nutrient dynamics in an anoxic/oxic process. , 2010, Bioresource technology.

[13]  Sylvie Gillot,et al.  The COST Simulation Benchmark: Description and Simulator Manual , 2001 .

[14]  N. B. Chang,et al.  Assessing wastewater reclamation potential by neural network model , 2003 .

[15]  Jorge Rodríguez,et al.  Rule-based diagnosis and supervision of a pilot-scale wastewater treatment plant using fuzzy logic techniques , 2002, Expert Syst. Appl..

[16]  Lluís A. Belanche Muñoz,et al.  Prediction of the bulking phenomenon in wastewater treatment plants , 2000, Artif. Intell. Eng..

[17]  Fernando Tadeo,et al.  Model-free learning control of neutralization processes using reinforcement learning , 2007, Eng. Appl. Artif. Intell..

[18]  Gustaf Olsson,et al.  Instrumentation, Control and Automation in Wastewater Systems , 2015 .

[19]  Ulf Jeppsson,et al.  Benchmark Simulation Model no. 1 (BSM1) , 2008 .

[20]  Wang Yan,et al.  Control rules of aeration in a submerged biofilm wastewater treatment process using fuzzy neural networks , 2009, Expert Syst. Appl..