Approximation of Boolean Functions by Local Search

Usually, local search methods are considered to be slow. In our paper, we present a simulated annealing-based local search algorithm for the approximation of Boolean functions with a proven time complexity that behaves relatively fast on randomly generated functions. The functions are represented by disjunctive normal forms (DNFs). Given a set of m uniformly distributed positive and negative examples of length n generated by a target function F(x1,...,xn), where the DNF consists of conjunctions with at most ℓ literals only, the algorithm computes with high probability a hypothesis H of length m · ℓ such that the error is zero on all examples. Our algorithm can be implemented easily and we obtained a relatively high percentage of correct classifications on test examples that were not presented in the learning phase. For example, for randomly generated F with n = 64 variables and a training set of m = 16384 examples, the error on the same number of test examples was about 19% on positive and 29% on negative examples, respectively. The proven complexity bound provides the basis for further studies on the average case complexity.

[1]  Winfried Hochstättler,et al.  Computational experience with general equilibrium problems , 1996, Comput. Optim. Appl..

[2]  O. Catoni.  Rough Large Deviation Estimates for Simulated Annealing: Application to Exponential Schedules , 1992 .

[3]  H. Shvaytser,et al.  Learnable and nonlearnable visual concepts , 1990 .

[4]  Bruce E. Hajek,et al.  Cooling Schedules for Optimal Annealing , 1988, Math. Oper. Res..

[5]  J. K. Lenstra,et al.  Local Search in Combinatorial Optimisation. , 1997 .

[6]  Giovanni Righini.  Annealing Algorithms for Multisource Absolute Location Problems on Graph , 1997, Comput. Optim. Appl..

[7]  Yishay Mansour,et al.  An O(nlog log n) learning algorithm for DNF under the uniform distribution , 1992, COLT '92.

[8]  Sjur Didrik Flåm,et al.  Learning Equilibrium Play: A Myopic Approach , 1999, Comput. Optim. Appl..

[9]  Karsten A. Verbeurgt.  Learning DNF under the uniform distribution in quasi-polynomial time , 1990, COLT '90.

[10]  R. Mooney.  Encouraging Experimental Results on Learning CNF , 1995 .

[11]  Andreas Alexander Albrecht,et al.  Stochastic Simulations of Two-Dimensional Composite Packings , 1997 .

[12]  D. Angluin.  Queries and Concept Learning , 1988 .

[13]  Emile H. L. Aarts,et al.  Simulated annealing and Boltzmann machines - a stochastic approach to combinatorial optimization and neural computing , 1990, Wiley-Interscience series in discrete mathematics and optimization.

[14]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[15]  V. Cerný.  Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm , 1985 .

[16]  Olivier Catoni,et al.  Metropolis, Simulated Annealing, and Iterated Energy Transformation Algorithms: Theory and Experiments , 1996, J. Complex..

[17]  Martin D. F. Wong,et al.  Floorplan design of VLSI circuits , 2005, Algorithmica.

[18]  Kristin P. Bennett,et al.  Multicategory Classification by Support Vector Machines , 1999, Comput. Optim. Appl..

[19]  Alberto L. Sangiovanni-Vincentelli,et al.  A theoretical framework for simulated annealing , 1991, Algorithmica.

[20]  L. Pitt,et al.  On the Learnability of Disjunctive Normal Form Formulas , 1995, Machine Learning.

[21]  M. Kearns,et al.  Recent Results on Boolean Concept Learning , 1987 .

[22]  Sigeru Omatu,et al.  Efficient Genetic Algorithms Using Simple Genes Exchange Local Search Policy for the Quadratic Assignment Problem , 2000, Comput. Optim. Appl..

[23]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[24]  Kristin P. Bennett,et al.  Feature minimization within decision trees , 1998 .

[25]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.