On Optimal Learning Algorithms for Multiplicity Automata

We study polynomial time learning algorithms for Multiplicity Automata (MA) and Multiplicity Automata Function (MAF) that minimize the access to one or more of the following resources: Equivalence queries, Membership queries or Arithmetic operations in the field F. This is in particular interesting when access to one or more of the above resources is significantly more expensive than the others. We apply new algebraic approach based on Matrix Theory to simplify the algorithms and the proofs of their correctness. We improve the arithmetic complexity of the problem and argue that it is almost optimal. Then we prove tight bound for the minimal number of equivalence queries and almost (up to log factor) tight bound for the number of membership queries.

[1]  Wolfgang Maass,et al.  Lower Bound Methods and Separation Results for On-Line Learning Models , 2004, Machine Learning.

[2]  Nader H. Bshouty,et al.  On Learning Programs and Small Depth Circuits , 1997, EuroCOLT.

[3]  Wolfgang Maass,et al.  Lower bound methods and separation results for on-line learning models , 1992, Machine Learning.

[4]  N. Elkies.  On finite sequences satisfying linear recursions , 2001, math/0105007.

[5]  Nader H. Bshouty,et al.  Learning Matrix Functions over Rings , 1997, EuroCOLT.

[6]  Don Coppersmith,et al.  Matrix multiplication via arithmetic progressions , 1987, STOC.

[7]  Francesco Bergadano,et al.  Learning Behaviors of Automata from Multiplicity and Equivalence Queries , 1994, SIAM J. Comput..

[8]  Amir Shpilka,et al.  Lower bounds for matrix product , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[9]  Markus Bläser.  Lower bounds for the multiplicative complexity of matrix multiplication , 1999, computational complexity.

[10]  Nader H. Bshouty,et al.  Learning Multivariate Polynomials from Substitution and Equivalence Queries , 1996, Electron. Colloquium Comput. Complex..

[11]  Francesco Bergadano,et al.  Learning Sat-k-DNF formulas from membership queries , 1996, STOC '96.

[12]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[13]  Eyal Kushilevitz,et al.  Learning Boxes in High Dimension , 1998, Algorithmica.

[14]  Tadao Kasami,et al.  A Polynomial Time Learning Algorithm for Recognizable Series , 1994 .

[15]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[16]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[17]  Francesco Bergadano,et al.  Learning behaviors of automata from shortest counterexamples , 1995, EuroCOLT.

[18]  Eyal Kushilevitz,et al.  Learning functions represented as multiplicity automata , 2000, JACM.

[19]  Francesco Bergadano,et al.  Learning Behaviors of Automata from Multiplicity and Equivalence Queries , 1996, SIAM J. Comput..

[20]  Osamu Watanabe,et al.  A Note on the Query Complexity of Learning DFA (Extended Abstract) , 1992, ALT.

[21]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[22]  Nader H. Bshouty,et al.  Asking questions to minimize errors , 1993, COLT '93.

[23]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.