Partially observable Markov decision processes for optimal operations of gas transmission networks

Abstract We develop a decision-support framework based on Partially Observable Markov Decision Processes (POMDPs) for the management of Gas Transmission Networks (GTNs) operations, encoding realistic degradation state estimations provided by Prognostics and Health Management (PHM) systems, while considering demand variations and the effects of the management decisions on the GTN degradation evolution. This Operation and Maintenance (O&M) framework allows optimally operating a GTN. Furthermore, the economic impact of using PHM systems with different accuracy levels can be estimated. The approach is shown with reference to a GTN of the literature.

[1]  Chris Sherlaw-Johnson,et al.  Estimating a Markov Transition Matrix from Observational Data , 1995 .

[2]  Sankalita Saha,et al.  Evaluating prognostics performance for algorithms incorporating uncertainty estimates , 2010, 2010 IEEE Aerospace Conference.

[3]  Jeffrey M. Keisler,et al.  Portfolio decision analysis : improved methods for resource allocation , 2011 .

[4]  Enrico Zio,et al.  Quantifying the reliability of fault classifiers , 2014, Inf. Sci..

[5]  Panos M. Pardalos,et al.  Optimization Models in The Natural Gas Industry , 2010 .

[6]  Paul A. Wawrzynek,et al.  Probabilistic fatigue damage prognosis using surrogate models trained via three-dimensional finite element analysis , 2017 .

[7]  Francesco Cadini,et al.  A particle filter‐based model selection algorithm for fatigue damage identification on aeronautical structures , 2017 .

[8]  Enrico Zio,et al.  Quantifying the importance of elements of a gas transmission network from topological, reliability and controllability perspectives, considering capacity constraints , 2017 .

[9]  D. Dopheide,et al.  Development of a New Fundamental Measuring Technique for the Accurate Measurement of Gas Flowrates by Means of Laser Doppler Anemometry , 1990 .

[10]  F. Gorucu.  Evaluation and Forecasting of Gas Consumption by Statistical Analysis , 2004 .

[11]  Enrico Zio,et al.  Computational Methods for Reliability and Risk Analysis , 2009 .

[12]  Alireza Kabirian,et al.  A strategic planning model for natural gas transmission networks , 2007 .

[13]  Paolo Toth,et al.  Optimization of a pipe-line for the natural gas transportation , 1982 .

[14]  Enrico Zio,et al.  A Markov decision process framework for optimal operation of monitored multi-state systems , 2018 .

[15]  Marie-Odile Cordier,et al.  Supply Restoration in Power Distribution Systems: A Case Study in Integrating Model-Based Diagnosis and Repair Planning , 1996, UAI.

[16]  Sankalita Saha,et al.  Metrics for Offline Evaluation of Prognostic Performance , 2021, International Journal of Prognostics and Health Management.

[17]  Aleksey Mironov,et al.  Condition Monitoring Of Operating Pipelines With Operational Modal Analysis Application , 2015 .

[18]  Enrico Zio,et al.  Monte Carlo-based filtering for fatigue crack growth estimation , 2009 .

[19]  Pavel Praks,et al.  Monte-Carlo Based Reliability Modelling of a Gas Network Using Graph Theory Approach , 2014, 2014 Ninth International Conference on Availability, Reliability and Security.

[20]  Michael L. Littman,et al.  Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes , 1997, UAI.

[21]  Shankar Sankararaman,et al.  Significance, interpretation, and quantification of uncertainty in prognostics and remaining useful life prediction , 2015 .

[22]  Milad Memarzadeh,et al.  Value of information in sequential decision making: Component inspection, permanent monitoring and system-level scheduling , 2016, Reliab. Eng. Syst. Saf..

[23]  William S. Lovejoy,et al.  Computationally Feasible Bounds for Partially Observed Markov Decision Processes , 1991, Oper. Res..

[24]  M. Shinozuka,et al.  Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part I: Theory , 2014, Reliab. Eng. Syst. Saf..

[25]  Joelle Pineau,et al.  Point-based value iteration: An anytime algorithm for POMDPs , 2003, IJCAI.

[26]  David E. Goldberg,et al.  Computer-aided pipeline operation using genetic algorithms and rule learning. PART I: Genetic algorithms in pipeline optimization , 1987, Engineering with Computers.

[27]  Patrick Fabiani,et al.  A Simulation-based Approach for Solving Generalized Semi-Markov Decision Processes , 2008, ECAI.

[28]  M. Shinozuka,et al.  Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation , 2014, Reliab. Eng. Syst. Saf..

[29]  A. Suomilammi,et al.  GAS COMPRESSOR UNIT PERFORMANCE MONITORING USING FUZZY CLUSTERING , 2006 .

[30]  Guy Shani,et al.  Noname manuscript No. (will be inserted by the editor) A Survey of Point-Based POMDP Solvers , 2022 .

[31]  Roger Z. Ríos-Mercado,et al.  Optimization problems in natural gas transportation systems. A state-of-the-art review , 2015 .

[32]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs , 1978, Oper. Res..

[33]  Alfred Inselberg,et al.  Parallel coordinates for visualizing multi-dimensional geometry , 1987 .

[34]  Pavel Praks,et al.  Monte-Carlo-based reliability and vulnerability assessment of a natural gas transmission system due to random network component failures , 2017 .

[35]  E. Kołodziński,et al.  Flow modelling in gas transmission networks. Part I – Mathematical model , 2002 .

[36]  J. Munoz,et al.  Natural gas network modeling for power systems reliability studies , 2003, 2003 IEEE Bologna Power Tech Conference Proceedings,.

[37]  Halit Üster,et al.  Optimization for Design and Operation of Natural Gas Transmission Networks , 2014 .

[38]  Anatoly Lisnianski,et al.  Multi-state Markov Model for Reliability Analysis of a Combined Cycle Gas Turbine Power Plant , 2016, 2016 Second International Symposium on Stochastic Models in Reliability Engineering, Life Science and Operations Management (SMRLO).

[39]  K. L. Lo.  Optimisation and analysis of gas supply networks , 1984 .

[40]  Hao Xu,et al.  An integrated approach for wind turbine gearbox fatigue life prediction considering instantaneously varying load conditions , 2018, Renewable Energy.

[41]  Nikos A. Vlassis,et al.  Perseus: Randomized Point-based Value Iteration for POMDPs , 2005, J. Artif. Intell. Res..

[42]  Enrico Zio,et al.  Availability Model of a PHM-Equipped Component , 2017, IEEE Transactions on Reliability.

[43]  Konstantinos Papakonstantinou,et al.  Optimum inspection and maintenance policies for corroded structures using partially observable Markov decision processes and stochastic, physically based models , 2014 .

[44]  Suming Wu,et al.  Model relaxations for the fuel cost minimization of steady-state gas pipeline networks , 2000 .

[45]  Yang Nan,et al.  An integrated systemic method for supply reliability assessment of natural gas pipeline networks , 2018 .

[46]  Enrico Zio,et al.  Portfolio decision analysis for risk-based maintenance of gas networks , 2019 .

[47]  Daniel Straub,et al.  Long-term adaption decisions via fully and partially observable Markov decision processes , 2017 .

[48]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  David Hsu,et al.  SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces , 2008, Robotics: Science and Systems.

[50]  Enrico Zio,et al.  Reliability model of a component equipped with PHM capabilities , 2017, Reliab. Eng. Syst. Saf..

[51]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[52]  Ahti Salo,et al.  Optimal strategies for selecting project portfolios using uncertain value estimates , 2014, Eur. J. Oper. Res..

[53]  Marcelo Masera,et al.  Probabilistic modelling of security of supply in gas networks and evaluation of new infrastructure , 2015, Reliab. Eng. Syst. Saf..

[54]  Luca A. Tagliafico,et al.  Scenario analysis of nonresidential natural gas consumption in Italy , 2014 .

[55]  Milos Hauskrecht,et al.  Value-Function Approximations for Partially Observable Markov Decision Processes , 2000, J. Artif. Intell. Res..

[56]  Enrico Zio,et al.  Homogeneous Continuous-Time, Finite-State Hidden Semi-Markov Modeling for Enhancing Empirical Classification System Diagnostics of Industrial Components , 2018 .

[57]  Stephen A. Holditch,et al.  Factors That Will Influence Oil and Gas Supply and Demand in the 21st Century , 2008 .

[58]  Ming Jian Zuo,et al.  An integrated framework for online diagnostic and prognostic health monitoring using a multistate deterioration process , 2014, Reliab. Eng. Syst. Saf..

[59]  Sankaran Mahadevan,et al.  Integration of structural health monitoring and fatigue damage prognosis , 2012 .