On the Convergence of Tsetlin Machines for the XOR Operator

The Tsetlin Machine (TM) is a novel machine learning algorithm with several distinct properties, including transparent inference and learning using hardware-near building blocks. Although numerous papers explore the TM empirically, many of its properties have not yet been analyzed mathematically. In this article, we analyze the convergence of the TM when input is non-linearly related to output by the XOR-operator. Our analysis reveals that the TM, with just two conjunctive clauses, can converge almost surely to reproducing XOR, learning from training data over an infinite time horizon. Furthermore, the analysis shows how the hyper-parameter T guides clause construction so that the clauses capture the distinct sub-patterns in the data. Our analysis of convergence for XOR thus lays the foundation for analyzing other more complex logical expressions. These analyses altogether, from a mathematical perspective, provide new insights on why TMs have obtained state-of-the-art performance on several pattern recognition problems.

[1]  Ole-Christoffer Granmo,et al.  Human-Level Interpretable Learning for Aspect-Based Sentiment Analysis , 2021, AAAI.

[2]  Ole-Christoffer Granmo,et al.  The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic , 2018, ArXiv.

[3]  Ole-Christoffer Granmo,et al.  The regression Tsetlin machine: a novel approach to interpretable nonlinear regression , 2019, Philosophical Transactions of the Royal Society A.

[4]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[5]  Ole-Christoffer Granmo,et al.  Closed-Form Expressions for Global and Local Interpretation of Tsetlin Machines with Applications to Explaining High-Dimensional Data , 2020, IEA/AIE.

[6]  Xuan Zhang,et al.  A Scheme for Continuous Input to the Tsetlin Machine with Applications to Forecasting Disease Outbreaks , 2019, IEA/AIE.

[7]  Xuan Zhang,et al.  A Conclusive Analysis of the Finite-Time Behavior of the Discretized Pursuit Learning Automaton , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Kuruge Darshana Abeyrathna,et al.  Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased Interpretability , 2020, IEEE Access.

[9]  Kuruge Darshana Abeyrathna,et al.  A Regression Tsetlin Machine with Integer Weighted Clauses for Compact Pattern Representation , 2020, ArXiv.

[10]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[11]  Ole-Christoffer Granmo,et al.  The Convolutional Tsetlin Machine , 2019, ArXiv.

[12]  Alex Yakovlev,et al.  Explainability and Dependability Analysis of Learning Automata based AI Hardware , 2020, 2020 IEEE 26th International Symposium on On-Line Testing and Robust System Design (IOLTS).

[13]  Ole-Christoffer Granmo,et al.  On the Convergence of Tsetlin Machines for the IDENTITY- and NOT Operators , 2021, IEEE transactions on pattern analysis and machine intelligence.

[14]  Cynthia Rudin,et al.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead , 2018, Nature Machine Intelligence.

[15]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[16]  Ole-Christoffer Granmo,et al.  Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization With Medical Applications , 2018, IEEE Access.