Error Analysis of NIST SP 800-22 Test Suite

Statistical tests for randomness play an essential role in cryptography, but the reliability of these tests is rarely taken into account, which may mislead our judgement of randomness especially with extremely large samples. In this paper, we make a two-stage error analysis of the commonly used two-level randomness tests in the NIST SP 800–22 test suite. Especially, we give the estimates of the ${p}$ -value deviations of chi-square approximation in the basic tests based on our proposed continuity constraints, and mathematically express the reliability of uniformity test used in the test suite with the fact of noncentral chi-square approximation. Finally, we analyze the respective error factors and derive the corresponding probability deviation estimation for the tests, explaining some false positive issues in practical test experiments. With our derived error analysis models, one can get a more reliable randomness test strategy with extremely large samples and wider parameter selections.

[1]  Ken Umeno,et al.  Randomness Test to Solve Discrete Fourier Transform Test Problems , 2018, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[2]  Atsushi Iwasaki,et al.  Deriving the Variance of the Discrete Fourier Transform Test Using Parseval’s Theorem , 2018, IEEE Transactions on Information Theory.

[3]  Onur Koçak,et al.  On the independence of statistical randomness tests included in the NIST test suite , 2017, Turkish J. Electr. Eng. Comput. Sci..

[4]  Ken Umeno,et al.  Randomness Evaluation With the Discrete Fourier Transform Test Based on Exact Analysis of the Reference Distribution , 2017, IEEE Transactions on Information Forensics and Security.

[5]  Yuan Ma,et al.  More Powerful and Reliable Second-Level Statistical Randomness Tests for NIST SP 800-22 , 2016, ASIACRYPT.

[6]  Robert E. Gaunt,et al.  Chi-square approximation by Stein's method with application to Pearson's statistic , 2015, 1507.01707.

[7]  Riccardo Rovatti,et al.  On Statistical Tests for Randomness Included in the NIST SP800-22 Test Suite and Based on the Binomial Distribution , 2012, IEEE Transactions on Information Forensics and Security.

[8]  Louis H. Y. Chen,et al.  Normal Approximation by Stein's Method , 2010 .

[9]  I. Tyurin.  New estimates of the convergence rate in the Lyapunov theorem , 2009, 0912.0726.

[10]  Toshinobu Kaneko,et al.  Correction of Overlapping Template Matching Test Included in NIST Randomness Test Suite , 2007, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[11]  Riccardo Rovatti,et al.  Second-level NIST Randomness Tests for Improving Test Reliability , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[12]  Pierre L'Ecuyer,et al.  TestU01: A C library for empirical testing of random number generators , 2006, TOMS.

[13]  Ken Umeno,et al.  Corrections of the NIST Statistical Test Suite for Randomness , 2004, IACR Cryptol. ePrint Arch..

[14]  Elaine B. Barker,et al.  A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications , 2000 .

[15]  Andrew L. Rukhin,et al.  Approximate entropy for testing randomness , 2000, Journal of Applied Probability.

[16]  E. Samuel-Cahn,et al.  P Values as Random Variables—Expected P Values , 1999 .

[17]  Jean-Sébastien Coron,et al.  An Accurate Evaluation of Maurer's Universal Test , 1998, Selected Areas in Cryptography.

[18]  Ronald H. Rnndles.  Nonparametric Statistical Inference (2nd ed.) , 1986 .

[19]  R. Randles,et al.  A Power Approximation for the Chi-Square Goodness-of-Fit Test: Simple Hypothesis Case , 1977 .

[20]  A. Mayne.  Generalized Inverse of Matrices and its Applications , 1972 .

[21]  J. Roscoe,et al.  An Investigation of the Restraints with Respect to Sample Size Commonly Imposed on the Use of the Chi-Square Statistic , 1971 .

[22]  K. Pearson.  On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can be Reasonably Supposed to have Arisen from Random Sampling , 1900 .

[23]  C. Georgescu,et al.  NEW RESULTS CONCERNING THE POWER OF NIST RANDOMNESS TESTS , 2017 .

[24]  A. M. Zubkov,et al.  Some remarks on the noncentral Pearson statistics distributions , 2016 .

[25]  K. Hamano.  The Distribution of the Spectrum for the Discrete Fourier Transform Test Included in SP800-22 , 2005, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..