Testing Optimality of Sequential Decision-Making

This paper provides a statistical method to test whether a system that performs a binary sequential hypothesis test is optimal in the sense of minimizing the average decision times while taking decisions with given reliabilities. The proposed method requires samples of the decision times, the decision outcomes, and the true hypotheses, but does not require knowledge on the statistics of the observations or the properties of the decision-making system. The method is based on fluctuation relations for decision time distributions which are proved for sequential probability ratio tests. These relations follow from the martingale property of probability ratios and hold under fairly general conditions. We illustrate these tests with numerical experiments and discuss potential applications.

[1]  Frank Jülicher,et al.  Generic Properties of Stochastic Entropy Production. , 2017, Physical review letters.

[2]  Christian Van den Broeck,et al.  Statistical Mechanics of Learning , 2001 .

[3]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[4]  Eric D. Siggia,et al.  Decisions on the fly in cellular sensory systems , 2013, Proceedings of the National Academy of Sciences.

[5]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[6]  R. Bogacz.  Optimal decision-making theories: linking neurobiology with behaviour , 2007, Trends in Cognitive Sciences.

[7]  L. Györfi,et al.  Nonparametric entropy estimation. An overview , 1997 .

[8]  A. V. D. Vaart,et al.  Asymptotic Statistics: U -Statistics , 1998 .

[9]  Frank Julicher,et al.  Statistics of Infima and Stopping Times of Entropy Production and Applications to Active Molecular Processes , 2016, 1604.04159.

[10]  Claude Desplan,et al.  Stochasticity and Cell Fate , 2008, Science.

[11]  Heinrich Meyr,et al.  Decision Making in the Arrow of Time. , 2015, Physical review letters.

[12]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[13]  W. R. Schucany,et al.  Generating Random Variates Using Transformations with Multiple Roots , 1976 .

[14]  A. Oudenaarden,et al.  Nature, Nurture, or Chance: Stochastic Gene Expression and Its Consequences , 2008, Cell.

[15]  Heinrich Meyr,et al.  An information theoretic analysis of sequential decision-making , 2015, 2017 IEEE International Symposium on Information Theory (ISIT).

[16]  Heinrich Meyr,et al.  Complete statistical description of the phase-error process generated by correlative tracking systems , 1977, IEEE Trans. Inf. Theory.

[17]  Alexander G. Tartakovsky.  Asymptotically optimal sequential tests for nonhomogeneous processes , 1998 .

[18]  Emre Ertin,et al.  Learning to Aggregate Information for Sequential Inferences , 2015, ArXiv.

[19]  J. Wolfowitz,et al.  Optimum Character of the Sequential Probability Ratio Test , 1948 .

[20]  Yanjun Han,et al.  Minimax Estimation of Functionals of Discrete Distributions , 2014, IEEE Transactions on Information Theory.

[21]  Kevin N. Gurney,et al.  The Basal Ganglia and Cortex Implement Optimal Decision Making Between Alternative Actions , 2007, Neural Computation.

[22]  Alexander G. Tartakovsky,et al.  Asymptotic Optimality of Certain Multihypothesis Sequential Tests: Non‐i.i.d. Case , 1998 .

[23]  Philip L. Smith,et al.  A comparison of sequential sampling models for two-choice reaction time. , 2004, Psychological review.

[24]  M. Basseville,et al.  Sequential Analysis: Hypothesis Testing and Changepoint Detection , 2014 .

[25]  M. Shadlen,et al.  A Neural Implementation of Wald’s Sequential Probability Ratio Test , 2015, Neuron.

[26]  Emre Ertin,et al.  Wald-Kernel: Learning to Aggregate Information for Sequential Inference , 2015 .

[27]  S. Redner.  A guide to first-passage processes , 2001 .

[28]  Venugopal V. Veeravalli,et al.  Multihypothesis sequential probability ratio tests - Part I: Asymptotic optimality , 1999, IEEE Trans. Inf. Theory.

[29]  J. L. Melsa,et al.  Decision and Estimation Theory , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[30]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[31]  T. Lai.  Asymptotic Optimality of Invariant Sequential Probability Ratio Tests , 1981 .

[32]  A. Shiryayev,et al.  Statistics of Random Processes I: General Theory , 1984 .

[33]  R. Khan,et al.  Sequential Tests of Statistical Hypotheses. , 1972 .

[34]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[35]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[36]  Anthony Kuh,et al.  Temporal difference learning applied to sequential detection , 1997, IEEE Trans. Neural Networks.