Heavy-Tailed Distributions in Combinatorial Search

Combinatorial search methods often exhibit a large variability in performance. We study the cost profiles of combinatorial search procedures. Our study reveals some intriguing properties of such cost profiles. The distributions are often characterized by very long tails or "heavy tails". We will show that these distributions are best characterized by a general class of distributions that have no moments (i.e., an infinite mean, variance, etc.). Such non-standard distributions have recently been observed in areas as diverse as economics, statistical physics, and geophysics. They are closely related to fractal phenomena, whose study was introduced by Mandelbrot. We believe this is the first finding of these distributions in a purely computational setting. We also show how random restarts can effectively eliminate heavy-tailed behavior, thereby dramatically improving the overall performance of a search procedure.

[1]  Alvin C. M. Kwan,et al.  Validity of Normality Assumption in CSP Research , 1996, PRICAI.

[2]  Tad Hogg,et al.  Using Deep Structure to Locate Hard Problems , 1992, AAAI.

[3]  P. D. Lima,et al.  On the robustness of nonlinearity tests to moment condition failure , 1997 .

[4]  Bart Selman,et al.  Problem Structure in the Presence of Perturbations , 1997, AAAI/IAAI.

[5]  Barbara M. Smith,et al.  Sparse Constraint Graphs and Exceptionally Hard Problems , 1995, IJCAI.

[6]  Weixiong Zhang,et al.  A Study of Complexity Transitions on the Asymmetric Traveling Salesman Problem , 1996, Artif. Intell..

[7]  L. Thiel,et al.  The Non-Existence of Finite Projective Planes of Order 10 , 1989, Canadian Journal of Mathematics.

[8]  D. Applebaum.  Stable non-Gaussian random processes , 1995, The Mathematical Gazette.

[9]  B. Mandelbrot.  THE PARETO-LEVY LAW AND THE DISTRIBUTION OF INCOME* , 1960 .

[10]  C. Mallows,et al.  A Method for Simulating Stable Random Variables , 1976 .

[11]  V. Zolotarev.  One-dimensional stable distributions , 1986 .

[12]  Toby Walsh,et al.  Easy Problems are Sometimes Hard , 1994, Artif. Intell..

[13]  Benoit B. Mandelbrot,et al.  Fractal Geometry of Nature , 1984 .

[14]  Tad Hogg,et al.  Phase Transitions and the Search Problem , 1996, Artif. Intell..

[15]  J. Dénes,et al.  Latin squares and their applications , 1974 .

[16]  Toby Walsh,et al.  The Satisfiability Constraint Gap , 1996, Artif. Intell..

[17]  P. Hall.  On Some Simple Estimates of an Exponent of Regular Variation , 1982 .

[18]  Eugene C. Freuder,et al.  Constraint-based reasoning , 1994 .

[19]  Bart Selman,et al.  Algorithm Portfolio Design: Theory vs. Practice , 1997, UAI.

[20]  W. Linde.  STABLE NON‐GAUSSIAN RANDOM PROCESSES: STOCHASTIC MODELS WITH INFINITE VARIANCE , 1996 .

[21]  B. Mandlebrot.  The Variation of Certain Speculative Prices , 1963 .

[22]  Hector J. Levesque,et al.  Hard and Easy Distributions of SAT Problems , 1992, AAAI.

[23]  Charles J. Colbourn,et al.  Embedding Partial Steiner Triple Systems Is NP-Complete , 1983, J. Comb. Theory, Ser. A.

[24]  Martin E. Dyer,et al.  Locating the Phase Transition in Binary Constraint Satisfaction Problems , 1996, Artif. Intell..

[25]  Peter C. Cheeseman,et al.  Where the Really Hard Problems Are , 1991, IJCAI.

[26]  Masayuki Fujita,et al.  Automatic Generation of Some Results in Finite Algebra , 1993, IJCAI.

[27]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[28]  Jean-francois Puget,et al.  A C++ implementation of CLP , 1997 .

[29]  Charles Vanden Eynden,et al.  Completing Partial Latin Squares , 1980, J. Comb. Theory, Ser. A.

[30]  Pascal Van Hentenryck,et al.  A Generic Arc-Consistency Algorithm and its Specializations , 1992, Artif. Intell..

[31]  S Kirkpatrick,et al.  Critical Behavior in the Satisfiability of Random Boolean Expressions , 1994, Science.

[32]  Bart Selman,et al.  Critical Behavior in the Computational Cost of Satisfiability Testing , 1996, Artif. Intell..

[33]  Daniel Brélaz,et al.  New methods to color the vertices of a graph , 1979, CACM.

[34]  Irina Rish,et al.  Summarizing CSP Hardness with Continuous Probability Distributions , 1997, AAAI/IAAI.

[35]  Tad Hogg,et al.  An Economics Approach to Hard Computational Problems , 1997, Science.

[36]  Jill C. D. S. Yaqub.  The non-existence of finite projective planes of Lenz-Barlotti class III 2 , 1967 .