A Predictive Approach to Nonparametric Inference for Adaptive Sequential Sampling of Psychophysical Experiments.

We present a predictive account on adaptive sequential sampling of stimulus-response relations in psychophysical experiments. Our discussion applies to experimental situations with ordinal stimuli when there is only weak structural knowledge available such that parametric modeling is no option. By introducing a certain form of partial exchangeability, we successively develop a hierarchical Bayesian model based on a mixture of Pólya urn processes. Suitable utility measures permit us to optimize the overall experimental sampling process. We provide several measures that are either based on simple count statistics or more elaborate information theoretic quantities. The actual computation of information theoretic utilities often turns out to be infeasible. This is not the case with our sampling method, which relies on an efficient algorithm to compute exact solutions of our posterior predictions and utility measures. Finally, we demonstrate the advantages of our framework on a hypothetical sampling problem.

[1]  A. Watson,et al.  Quest: A Bayesian adaptive psychometric method , 1983, Perception & psychophysics.

[2]  M. Leek.  Adaptive procedures in psychophysical research , 2001, Perception & psychophysics.

[3]  Donald A. Berry,et al.  Simulation-based sequential Bayesian design , 2007 .

[4]  B. D. Finetti.  La prévision : ses lois logiques, ses sources subjectives , 1937 .

[5]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[6]  G. Pólya,et al.  Über die Statistik verketteter Vorgänge , 1923 .

[7]  Mark A. Pitt,et al.  Adaptive Design Optimization: A Mutual Information-Based Approach to Model Discrimination in Cognitive Science , 2010, Neural Computation.

[8]  Steffen L. Lauritizen.  On the Interrelationships Among Sufficiency, Total Sufficiency and Some Related Concepts , 1974 .

[9]  H. Levitt.  Transformed up-down methods in psychoacoustics. , 1971, The Journal of the Acoustical Society of America.

[10]  S. McKee,et al.  Statistical properties of forced-choice psychometric functions: Implications of probit analysis , 1985, Perception & psychophysics.

[11]  J. Hartigan,et al.  Product Partition Models for Change Point Problems , 1992 .

[12]  Lee J. Cronbach,et al.  A CONSIDERATION OF INFORMATION THEORY AND UTILITY THEORY AS TOOLS FOR PSYCHOMETRIC PROBLEMS , 1953 .

[13]  C. Tyler,et al.  Bayesian adaptive estimation of psychometric slope and threshold , 1999, Vision Research.

[14]  Tuomas J. Lukka,et al.  Bayesian adaptive estimation: The next dimension , 2006 .

[15]  J. Bernardo.  Expected Information as Expected Utility , 1979 .

[16]  M. Degroot.  Uncertainty, Information, and Sequential Experiments , 1962 .

[17]  Yi-Ching Yao.  Estimation of a Noisy Discrete-Time Step Function: Bayes and Empirical Bayes Approaches , 1984 .

[18]  M. M. Taylor,et al.  PEST: Efficient Estimates on Probability Functions , 1967 .

[19]  Catherine Tallon-Baudry,et al.  Relational information in visual short-term memory: the structural gist. , 2005, Journal of vision.

[20]  Peter Földiák,et al.  Bayesian bin distribution inference and mutual information , 2005, IEEE Transactions on Information Theory.

[21]  Marcus Hutter.  Exact Bayesian regression of piecewise constant functions , 2007 .

[22]  Janne V. Kujala.  Obtaining the best value for money in adaptive sequential estimation , 2010 .

[23]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[24]  Frank Jäkel,et al.  Bayesian inference for psychometric functions. , 2005, Journal of vision.

[25]  Paul Fearnhead,et al.  Exact and efficient Bayesian inference for multiple changepoint problems , 2006, Stat. Comput..

[26]  D. Lindley.  On a Measure of the Information Provided by an Experiment , 1956 .

[27]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[28]  Tobias Elze,et al.  Chinese characters reveal impacts of prior experience on very early stages of perception , 2011, BMC Neuroscience.

[29]  Peter Földiák,et al.  Bayesian binning beats approximate alternatives: estimating peri-stimulus time histograms , 2007, NIPS.

[30]  Seymour Geisser,et al.  8. Predictive Inference: An Introduction , 1995 .

[31]  A. Watson,et al.  The method of constant stimuli is inefficient , 1990, Perception & psychophysics.

[32]  F A Wichmann,et al.  Ning for Helpful Comments and Suggestions. This Paper Benefited Con- Siderably from Conscientious Peer Review, and We Thank Our Reviewers the Psychometric Function: I. Fitting, Sampling, and Goodness of Fit , 2001 .