Uncertainty Analysis in Well Log Classification by Bayesian Long Short-term Memory Networks

ABSTRACT As an important parameter, uncertainty can provide essential information in decision making, which is usually discarded in a typical application of deep-learning methods. Dropout, a technique to reduce overfitting and co-adaption in hidden neurons, is proposed to approximate the Bayesian inference realized in deep learning. Specifically, the dropout scheme is applied not only in the training process, but also in the testing step by the trained model. This novel approach would then enable the learned networks to express uncertainty via their own parameters. The proposed method is implemented in a recurrent system, particularly, long short-term memory (LSTM) to account for the data dependency for the classification of sequential lithologies in the subsurface. The designed Bayesian LSTM (BLSTM) is applied to a real data set from the North Sea. Different combinations of dropout ratio are assigned in the weight matrices, to test its influence on the classification performance. To simulate an ensemble predictor, random samples are drawn from the trained networks according to the pre-defined dropout ratio during the classification process, and the model average is regarded as the final classification result. The multi-class predictive entropy can provide the uncertainty information, and its lower value is associated with more confident prediction or vice versa. The traditional LSTM is applied as a benchmark, which yields a same good quality performance with BLSTM, as measured by the Matthews correlation coefficient and confusion matrix; however, it lacks the capability to permit analyzing the uncertainty probabilistically.