Extraction of optimally unbiased bits from a biased source

We explore the problem of transforming n independent and identically biased {-1,1}-valued random variables X/sub 1/,...,X/sub n/ into a single {-1,1} random variable f(X/sub 1/,...,X/sub n/), so that this result is as unbiased as possible. In general, no function f produces a completely unbiased result. We perform the first study of the relationship between the bias b of these X/sub i/ and the rate at which f(X/sub 1/,...,X/sub n/) can converge to an unbiased {-1,1} random variable (as n/spl rarr//spl infin/). A {-1,1} random variable has bias b if E(X/sub i/)=b. Fixing a bias b, we explore the rate at which the output bias |E(f(X/sub 1/,...,X/sub n/))| can tend to zero for a function f:{-1,1}*/spl rarr/{-1,1}. This is accomplished by classifying the behavior of the natural normalized quantity /spl Xi/(b)/spl Delta/inf/sub f/[lim/sub n/spl rarr//spl infin//n/spl radic/(|E(f(X/sub 1/,...,X/sub n/))|] this infimum taken over all such f. We show that for rational b, /spl Xi/(b)=(1/s), where (1+b/2)=(r/s) (r and s relatively prime). Developing the theory of uniform distribution of sequences to suit our problem, we then explore the case where b is irrational. We prove a new metrical theorem concerning multidimensional Diophantine approximation type from which we show that for (Lebesgue) almost all biases b, /spl Xi/(b)=0. Finally, we show that algebraic biases exhibit curious "boundary" behavior, falling into two classes. Class 1. Those algebraics b for which /spl Xi/(b)>0 and, furthermore, c/sub 1//spl les//spl Xi/(b)/spl les/c/sub 2/ where c/sub 1/ and c/sub 2/ are positive constants depending only on b's algebraic characteristics. Class 2. Those algebraics b for which there exist n>0 and f: {-1,1}/sup n//spl rarr/{-1,1} so that E(f(X/sub 1/,...,X/sub n/))=0. Notice that this classification excludes the possibility that n/spl radic/(|E(f(X/sub 1/,...,X/sub n/))| limits to zero (for algebraics). For rational and algebraic biases, we also study the computational problem by restricting f to be a polynomial time computable function. Finally, we discuss natural extensions where output distributions other than the uniform distribution on {-1,1} are sought.

[1]  Paul A. Samuelson.  Constructing an Unbiased Random Sequence , 1968 .

[2]  Wolfgang M. Schmidt,et al.  A Metrical Theorem in Diophantine Approximation , 1960, Canadian Journal of Mathematics.

[3]  Manuel Blum.  Independent unbiased coin flips from a correlated biased source—A finite state markov chain , 1986, Comb..

[4]  T. Cover.  On Determining the Irrationality of the Mean of a Random Variable , 1973 .

[5]  N. S. Barnett,et al.  Private communication , 1969 .

[6]  Toshiya Itoh.  Simulating Fair Dice with Biased Coins , 1996, Inf. Comput..

[7]  Wassily Hoeffding,et al.  UNBIASED COIN TOSSING WITH A BIASED COIN by , 1969 .

[8]  Harald Niederreiter,et al.  Methods for Estimating Discrepancy , 1972 .

[9]  W. Hoeffding,et al.  Unbiased Coin Tossing With a Biased Coin , 1970 .

[10]  William W. Adams,et al.  Simultaneous asymptotic Diophantine approximations , 1967 .

[11]  Wolfgang M. Schmidt,et al.  Simultaneous approximation to algebraic numbers by rationals , 1970 .

[12]  David Zuckerman,et al.  General weak random sources , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[13]  Oded Goldreich,et al.  Unbiased Bits from Sources of Weak Randomness and Probabilistic Communication Complexity , 1988, SIAM J. Comput..

[14]  Lauwerens Kuipers,et al.  Uniform distribution of sequences , 1974 .

[15]  Jean-Baptiste Lully,et al.  The collected works , 1996 .

[16]  S. Lang,et al.  Introduction to Diophantine Approximations , 1995 .

[17]  Quentin F. Stout,et al.  TREE ALGORITHMS FOR UNBIASED COIN TOSSING WITH A BIASED COIN , 1984 .

[18]  József Beck,et al.  Probabilistic diophantine approximation, I. Kronecker sequences , 1994 .

[19]  Meyer Dwass.  Unbiased Coin Tossing with Discrete Random Variables , 1972 .

[20]  Miklos Santha,et al.  Generating Quasi-random Sequences from Semi-random Sources , 1986, J. Comput. Syst. Sci..

[21]  Miklos Santha,et al.  Generating Quasi-Random Sequences from Slightly-Random Sources (Extended Abstract) , 1984, FOCS.

[22]  Joel Friedman,et al.  On the bit extraction problem , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[23]  Moni Naor,et al.  On Dice and Coins: Models of Computation for Random Generation , 1993, Inf. Comput..

[24]  Noga Alon,et al.  Biased Coins and Randomized Algorithms , 1989, Advances in Computational Research.

[25]  Aravind Srinivasan,et al.  Computing with very weak random sources , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[26]  Andrew Chi-Chih Yao,et al.  The complexity of nonuniform random number generation , 1976 .

[27]  P. Elias.  The Efficient Construction of an Unbiased Random Sequence , 1972 .

[28]  Vijay V. Vazirani,et al.  Random polynomial time is equal to slightly-random polynomial time , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).