Using an Ant Colony Optimization Algorithm for Monotonic Regression Rule Discovery

Many data mining algorithms do not make use of existing domain knowledge when constructing their models. This can lead to model rejection as users may not trust models that behave contrary to their expectations. Semantic constraints provide a way to encapsulate this knowledge which can then be used to guide the construction of models. One of the most studied semantic constraints in the literature is monotonicity, however current monotonically-aware algorithms have focused on ordinal classification problems. This paper proposes an extension to an ACO-based regression algorithm in order to extract a list of monotonic regression rules. We compared the proposed algorithm against a greedy regression rule induction algorithm that preserves monotonic constraints and the well-known M5' Rules. Our experiments using eight publicly available data sets show that the proposed algorithm successfully creates monotonic rules while maintaining predictive accuracy.

[1]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery: An Overview , 1996, Advances in Knowledge Discovery and Data Mining.

[2]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[3]  A. J. Feelders,et al.  Pruning for Monotone Classification Trees , 2003, IDA.

[4]  Fernando E. B. Otero,et al.  Discovering Regression Rules with Ant Colony Optimization , 2015, GECCO.

[5]  Johannes Fürnkranz,et al.  Separate-and-conquer Regression , 2010, LWA.

[6]  A. J. Feelders,et al.  Nearest Neighbour Classification with Monotonicity Constraints , 2008, ECML/PKDD.

[7]  Alex Alves Freitas,et al.  cAnt-Miner: An Ant Colony Classification Algorithm to Cope with Continuous Attributes , 2008, ANTS Conference.

[8]  Helio J. C. Barbosa,et al.  Symbolic regression via genetic programming , 2000, Proceedings. Vol.1. Sixth Brazilian Symposium on Neural Networks.

[9]  A. Ben-David.  Monotonicity Maintenance in Information-Theoretic Machine Learning Algorithms , 1995, Machine Learning.

[10]  Ludwig Fahrmeir,et al.  Regression: Models, Methods and Applications , 2013 .

[11]  Geoff Holmes,et al.  Generating Rule Sets from Model Trees , 1999, Australian Joint Conference on Artificial Intelligence.

[12]  Bart Baesens,et al.  Building Acceptable Classification Models , 2010, Data Mining.

[13]  Alex Alves Freitas,et al.  A New Sequential Covering Strategy for Inducing Classification Rules With Ant Colony Algorithms , 2013, IEEE Transactions on Evolutionary Computation.

[14]  Michiel C. van Wezel,et al.  Two algorithms for generating structured and unstructured monotone ordinal data sets , 2009, Eng. Appl. Artif. Intell..

[15]  Bart Baesens,et al.  Ant-Based Approach to the Knowledge Fusion Problem , 2006, ANTS Workshop.

[16]  J. R. Quinlan.  Learning With Continuous Classes , 1992 .

[17]  Leon Sterling,et al.  Adding monotonicity to learning algorithms may impair their accuracy , 2009, Expert Syst. Appl..

[18]  Jiye Liang,et al.  Fusing Monotonic Decision Trees , 2015, IEEE Transactions on Knowledge and Data Engineering.

[19]  Stephen J. Perez,et al.  Three attitudes towards data mining , 2000 .

[20]  H. Daniels,et al.  Derivation of monotone decision models from noisy data , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[21]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[22]  Alex Alves Freitas,et al.  Data mining with an ant colony optimization algorithm , 2002, IEEE Trans. Evol. Comput..