Bayesian measures of surprise for outlier detection

From a Bayesian point of view, testing whether an observation is an outlier is usually reduced to a testing problem concerning a parameter of a contaminating distribution. This requires elicitation of both (i) the contaminating distribution that generates the outlier and (ii) prior distributions on its parameters. However, very little information is typically available about how the possible outlier could have been generated. Thus easy, preliminary checks in which these assessments can often be avoided may prove useful. Several such measures of surprise are derived for outlier detection in normal models. Results are applied to several examples. Default Bayes factors, where the contaminating model is assessed but not the prior distribution, are also computed.

[1]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[2]  L. I. Pettit,et al.  Bayes methods for outliers in exponential samples , 1988 .

[3]  Anthony O'Hagan,et al.  Kendall's Advanced Theory of Statistics: Vol. 2B, Bayesian Inference. , 1996 .

[4]  A. O'Hagan,et al.  Fractional Bayes factors for model comparison , 1995 .

[5]  Michael Evans,et al.  Bayesian ikference procedures derived via the concept of relative surprise , 1997 .

[6]  A. O'Hagan,et al.  On Outlier Rejection Phenomena in Bayes Inference , 1979 .

[7]  M. Aitkin.  Posterior Bayes Factors , 1991 .

[8]  J. Berger,et al.  Expected‐posterior prior distributions for model selection , 2002 .

[9]  Bovas Abraham,et al.  Linear Models and Spurious Observations , 1978 .

[10]  George E. P. Box,et al.  Deterministic and Forecast‐Adaptive Time‐Dependent Models , 1978 .

[11]  M. J. Bayarri,et al.  Calibration of ρ Values for Testing Precise Null Hypotheses , 2001 .

[12]  F. E. Grubbs.  Procedures for Detecting Outlying Observations in Samples , 1969 .

[13]  L. Wasserman,et al.  Bayesian analysis of outlier problems using the Gibbs sampler , 1991 .

[14]  Arnold Zellner,et al.  Bayesian Analysis of Regression Error Terms , 1975 .

[15]  A. Dawid.  Posterior expectations for large observations , 1973 .

[16]  M. West.  Outlier Models and Prior Distributions in Bayesian Linear Regression , 1984 .

[17]  Vic Barnett,et al.  Outliers in Statistical Data , 1980 .

[18]  Predictive discordancy tests for exponential observations , 1989 .

[19]  I. Guttman.  Care and Handling of Univariate or Multivariate Outliners in Detecting Spuriosity—a Bayesian Approach , 1973 .

[20]  George E. P. Box,et al.  Sampling and Bayes' inference in scientific modelling and robustness , 1980 .

[21]  Anthony O'Hagan,et al.  Outliers and Credence for Location Parameter Inference , 1990 .

[22]  Rudolf Dutter,et al.  Care and Handling of Univariate Outliers in the General Linear Model to Detect Spuriosity- A , 1978 .

[23]  D. Madigan,et al.  A method for simultaneous variable selection and outlier identification in linear regression , 1996 .

[24]  P. Freeman.  On the number of outliers in data from a linear model , 1980 .

[25]  Bruno De Finetti,et al.  The Bayesian Approach to the Rejection of Outliers , 1961 .

[26]  W. B. Gallie,et al.  Uncertainty and Business Decisions. , 1954 .

[27]  D. Spiegelhalter,et al.  Bayes Factors for Linear and Log‐Linear Models with Vague Prior Information , 1982 .

[28]  James M. Robins,et al.  Asymptotic Distribution of P Values in Composite Null Models , 2000 .

[29]  W. Weaver,et al.  Probability, rarity, interest, and surprise. , 1948, The Scientific monthly.

[30]  D. Rubin.  Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician , 1984 .

[31]  Irving John Good,et al.  The Surprise Index for the Multivariate Normal Distribution , 1956 .

[32]  R. Weiss.  An approach to Bayesian sensitivity analysis , 1996 .

[33]  Eric T. Bradlow,et al.  Case Influence Analysis in Bayesian Inference , 1997 .

[34]  K. Chaloner,et al.  A Bayesian approach to outlier detection and residual analysis , 1988 .

[35]  V. Yohai,et al.  A Fast Procedure for Outlier Diagnostics in Large Regression Problems , 1999 .

[36]  L. I. Pettit,et al.  Bayes Factors for Outlier Models Using the Device of Imaginary Observations , 1992 .

[37]  Seymour Geisser,et al.  Influential observations, diagnostics and discovery tests , 1987 .

[38]  Hong Chang,et al.  Model Determination Using Predictive Distributions with Implementation via Sampling-Based Methods , 1992 .

[39]  I. Guttman.  The Use of the Concept of a Future Observation in Goodness‐Of‐Fit Problems , 1967 .

[40]  J. Berger,et al.  The Intrinsic Bayes Factor for Model Selection and Prediction , 1996 .

[41]  J. Berger,et al.  Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence , 1987 .

[42]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[43]  Eric R. Ziecel.  Aspects of Uncertainty , 1995 .

[44]  L. Pettit.  The Conditional Predictive Ordinate for the Normal Distribution , 1990 .

[45]  G. C. Tiao,et al.  A bayesian approach to some outlier problems. , 1968, Biometrika.

[46]  Xiao-Li Meng,et al.  Posterior Predictive $p$-Values , 1994 .

[47]  Xiao-Li Meng,et al.  POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES , 1996 .

[48]  M. J. Bayarri,et al.  P Values for Composite Null Models , 2000 .

[49]  Warren Weaver,et al.  Lady Luck: The Theory of Probability. , 1965 .