Argument based machine learning

We present a novel approach to machine learning, called ABML (argumentation based ML). This approach combines machine learning from examples with concepts from the field of argumentation. The idea is to provide expert's arguments, or reasons, for some of the learning examples. We require that the theory induced from the examples explains the examples in terms of the given reasons. Thus arguments constrain the combinatorial search among possible hypotheses, and also direct the search towards hypotheses that are more comprehensible in the light of expert's background knowledge. In this paper we realize the idea of ABML as rule learning. We implement ABCN2, an argument-based extension of the CN2 rule learning algorithm, conduct experiments and analyze its performance in comparison with the original CN2 algorithm.

[1]  Timothy Chklovski,et al.  Using analogy to acquire commonsense knowledge from human Contributors , 2003 .

[2]  Christina Gloeckner,et al.  Modern Applied Statistics With S , 2003 .

[3]  Igor Kononenko,et al.  Inductive and Bayesian learning in medical diagnosis , 1993, Appl. Artif. Intell..

[4]  Bogdan E. Popescu,et al.  PREDICTIVE LEARNING VIA RULE ENSEMBLES , 2008, 0811.1679.

[5]  Steffen Staab,et al.  Discovering Conceptual Relations from Text , 2000, ECAI.

[6]  Peter A. Flach,et al.  Rule Evaluation Measures: A Unifying View , 1999, ILP.

[7]  B. Brookes,et al.  Statistical Theory of Extreme Values and Some Practical Applications , 1955, The Mathematical Gazette.

[8]  Blaz Zupan,et al.  A function-decomposition method for development of hierarchical multi-attribute decision models , 2004, Decis. Support Syst..

[9]  K. Menninger Law , 1984, Encyclopedia of Autism Spectrum Disorders.

[10]  Sunil J Rao,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2003 .

[11]  Ivan Bratko,et al.  Why Is Rule Learning Optimistic and How to Correct It , 2006, ECML.

[12]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[13]  Trevor J. M. Bench-Capon,et al.  Application of argument based machine learning to law , 2005, ICAIL '05.

[14]  Ivan Bratko,et al.  On Estimating Probabilities in Tree Pruning , 1991, EWSL.

[15]  Michael I. Jordan,et al.  The Handbook of Brain Theory and Neural Networks , 2002 .

[16]  Ivan Bratko,et al.  Prolog Programming for Artificial Intelligence , 1986 .

[17]  Herbert A. Simon,et al.  Applications of machine learning and rule induction , 1995, CACM.

[18]  Claudio Giuliano,et al.  Relation extraction and the influence of automatic named-entity recognition , 2007, TSLP.

[19]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[20]  William W. Cohen Grammatically Biased Learning: Learning Logic Programs Using an Explicit Antecedent Description Language , 1994, Artif. Intell..

[21]  Kyuseok Shim,et al.  Mining Sequential Patterns with Regular Expression Constraints , 2002, IEEE Trans. Knowl. Data Eng..

[22]  R. Mike Cameron-Jones,et al.  Oversearching and Layered Search in Empirical Learning , 1995, IJCAI.

[23]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[24]  S. Toulmin The uses of argument , 1960 .

[25]  Johannes Fürnkranz,et al.  Pruning Algorithms for Rule Learning , 1997, Machine Learning.

[26]  Blaz Zupan,et al.  Orange: From Experimental Machine Learning to Interactive Data Mining , 2004, PKDD.

[27]  Eric P. Smith,et al.  An Introduction to Statistical Modeling of Extreme Values , 2002, Technometrics.

[28]  Pedro M. Domingos The Role of Occam's Razor in Knowledge Discovery , 1999, Data Mining and Knowledge Discovery.

[29]  Ivan Bratko,et al.  Automated Chess Tutor , 2006, Computers and Games.

[30]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[31]  Ljup Co Todorovski Declarative Bias in Equation Discovery , 1997 .

[32]  M. Pazzani,et al.  The Utility of Knowledge in Inductive Learning , 1992, Machine Learning.

[33]  William W. Cohen Fast Effective Rule Induction , 1995, ICML.

[34]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[35]  Michael J. Pazzani,et al.  Beyond Concise and Colorful: Learning Intelligible Rules , 1997, KDD.

[36]  Gerard Vreeswijk,et al.  Abstract Argumentation Systems , 1997, Artif. Intell..

[37]  Saso Dzeroski,et al.  Inductive Logic Programming: Techniques and Applications , 1993 .

[38]  Sebastian Thrun,et al.  Integrating Inductive Neural Network Learning and Explanation-Based Learning , 1993, IJCAI.

[39]  Martin Mozina,et al.  Nomograms for Visualization of Naive Bayesian Classifier , 2004, PKDD.

[40]  Tom M. Mitchell,et al.  Explanation-Based Generalization: A Unifying View , 1986, Machine Learning.

[41]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[42]  Kamal Nigamyknigam,et al.  Employing Em in Pool-based Active Learning for Text Classiication , 1998 .

[43]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[44]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[45]  Lilian Edwards,et al.  Modelling law using a feminist theoretical perspective 1 , 1995 .

[46]  John H. Boose,et al.  A survey of knowledge acquisition techniques and tools , 1993 .

[47]  Paul R. Cohen,et al.  Multiple Comparisons in Induction Algorithms , 2000, Machine Learning.

[48]  Ivan Bratko,et al.  Fighting Knowledge Acquisition Bottleneck with Argument Based Machine Learning , 2008, ECAI.

[49]  Kevin D. Ashley,et al.  Law, learning and representation , 2003, Artif. Intell..

[50]  Henry Prakken,et al.  Logics for Defeasible Argumentation , 2001 .

[51]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[52]  R. Mike Cameron-Jones,et al.  Induction of logic programs: FOIL and related systems , 1995, New Generation Computing.

[53]  Chris Reed,et al.  Argumentation Machines, New Frontiers in Argument and Computation , 2004, Argumentation Machines.

[54]  Guillermo Ricardo Simari,et al.  A Mathematical Treatment of Defeasible Reasoning and its Implementation , 1992, Artif. Intell..

[55]  Ramakrishnan Srikant,et al.  Mining Association Rules with Item Constraints , 1997, KDD.

[56]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[57]  J. Rossouw,et al.  Coronary risk factor screening in three rural communities. The CORIS baseline study. , 1983, South African medical journal = Suid-Afrikaanse tydskrif vir geneeskunde.

[58]  Ivan Bratko,et al.  Learning Positional Features for Annotating Chess Games: A Case Study , 2008, Computers and Games.

[59]  Geoffrey I. Webb,et al.  An Experimental Evaluation of Integrating Machine Learning with Knowledge Acquisition , 1999, Machine Learning.

[60]  Peter Clark,et al.  Rule Induction with CN2: Some Recent Improvements , 1991, EWSL.

[61]  Roy Rada,et al.  Machine learning - applications in expert systems and information retrieval , 1986, Ellis Horwood series in artificial intelligence.

[62]  Peter Clark,et al.  Representing Arguments as Background Knowledge for Constraining Gereralisation , 1988, EWSL.

[63]  Ted Dunning,et al.  Accurate Methods for the Statistics of Surprise and Coincidence , 1993, CL.

[64]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[65]  Trevor J. M. Bench-Capon,et al.  An experiment in discovering association rules in the legal domain , 2000, Proceedings 11th International Workshop on Database and Expert Systems Applications.

[66]  David Stuart Robertson,et al.  Argument-based applications to knowledge engineering , 2000, The Knowledge Engineering Review.

[67]  Ulrich Rückert,et al.  A statistical approach to rule learning , 2006, ICML.

[68]  Mark Stevenson,et al.  A Semantic Approach to IE Pattern Induction , 2005, ACL.

[69]  Andrew McCallum,et al.  Toward Optimal Active Learning through Sampling Estimation of Error Reduction , 2001, ICML.

[70]  William A. Gale,et al.  A sequential algorithm for training text classifiers , 1994, SIGIR '94.

[71]  John R. Anderson,et al.  The role of background knowledge in sentence processing , 2001 .

[72]  Paul D. Allopenna,et al.  The locus of knowledge effects in concept learning. , 1994, Journal of experimental psychology. Learning, memory, and cognition.

[73]  Sašo Džeroski,et al.  Using the m -estimate in rule induction , 1993 .

[74]  Anil K. Jain,et al.  Clustering with Soft and Group Constraints , 2004, SSPR/SPR.

[75]  Qiang Sun,et al.  Explanation-Augmented SVM: an approach to incorporating domain knowledge into SVM learning , 2005, ICML.

[76]  Igor Kononenko,et al.  Machine Learning and Data Mining: Introduction to Principles and Algorithms , 2007 .

[77]  R. Fisher,et al.  Limiting forms of the frequency distribution of the largest or smallest member of a sample , 1928, Mathematical Proceedings of the Cambridge Philosophical Society.

[78]  Peter A. Flach,et al.  Subgroup Discovery with CN2-SD , 2004, J. Mach. Learn. Res..

[79]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[80]  Carlos Iván Chesñevar,et al.  Integrating defeasible argumentation with fuzzy ART neural networks for pattern classification , 2004 .

[81]  Eibe Frank,et al.  Logistic Model Trees , 2003, Machine Learning.

[82]  Mark Craven,et al.  An Analysis of Active Learning Strategies for Sequence Labeling Tasks , 2008, EMNLP.

[83]  Ivan Bratko,et al.  Rectifying Predictions of Classifiers by Local Rules , 2008 .

[84]  Johannes Fürnkranz,et al.  ROC ‘n’ Rule Learning—Towards a Better Understanding of Covering Algorithms , 2005, Machine Learning.

[85]  Sholom M. Weiss,et al.  Lightweight Rule Induction , 2000, ICML.

[86]  Dov M. Gabbay,et al.  Handbook of Philosophical Logic , 2002 .

[87]  Stan Matwin,et al.  Using Qualitative Models to Guide Inductive Learning , 1993, ICML.

[88]  Guido Governatori,et al.  Induction of defeasible logic theories in the legal domain , 2003, ICAIL.

[89]  Claire Cardie,et al.  Proceedings of the Eighteenth International Conference on Machine Learning, 2001, p. 577–584. Constrained K-means Clustering with Background Knowledge , 2022 .

[90]  Y. Hochberg A sharper Bonferroni procedure for multiple tests of significance , 1988 .

[91]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[92]  Wojciech Kotlowski,et al.  Maximum likelihood rule ensembles , 2008, ICML '08.

[93]  Marlon Núñez The use of background knowledge in decision tree induction , 2004, Machine Learning.

[94]  Edward A. Feigenbaum,et al.  Some challenges and grand challenges for computational intelligence , 2003, JACM.

[95]  Ivan Bratko,et al.  Argument Based Machine Learning in a Medical Domain , 2006, COMMA.

[96]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[97]  Saso Dzeroski,et al.  Theory Revision in Equation Discovery , 2001, Discovery Science.

[98]  Trevor J. M. Bench-Capon Knowledge-based systems and legal applications , 1991 .

[99]  John L. Pollock,et al.  How to Reason Defeasibly , 1992, Artif. Intell..

[100]  Ivan Bratko,et al.  Improving Numerical Prediction with Qualitative Constraints , 2003, ECML.

[101]  Tony Lindgren Methods for Rule Conflict Resolution , 2004, ECML.

[102]  David Madigan,et al.  Large-Scale Bayesian Logistic Regression for Text Categorization , 2007, Technometrics.

[103]  Wray L. Buntine,et al.  Interactive induction , 1991, [1988] Proceedings. The Fourth Conference on Artificial Intelligence Applications.

[104]  Martin Mozina,et al.  Predictive model for estimating risk of crush syndrome: a data mining approach. , 2007, The Journal of trauma.

[105]  N. Lavra,et al.  Predictive Performance of Weighted Relative Accuracy , 2000 .

[106]  Bernhard Schölkopf,et al.  Prior Knowledge in Support Vector Kernels , 1997, NIPS.

[107]  Bojan Cestnik,et al.  Estimating Probabilities: A Crucial Task in Machine Learning , 1990, ECAI.

[108]  C. Fellbaum An Electronic Lexical Database , 1998 .

[109]  Kevin D. Ashley,et al.  Predicting outcomes of case based legal arguments , 2003, ICAIL.

[110]  Douglas Walton,et al.  Fundamentals of critical argumentation , 2006, Critical reasoning and argumentation.

[111]  H. Richardson Practical Reasoning about Final Ends , 1994 .

[112]  Jeremy Roschelle,et al.  Learning in Interactive Environments: Prior Knowledge and New Experience , 2007 .

[113]  Ivan Bratko,et al.  Arguments Extracted from Text in Argument Based Machine Learning : A Case Study , 2009 .

[114]  Arnold W. M. Smeulders,et al.  Active learning using pre-clustering , 2004, ICML.

[115]  E A Feigenbaum,et al.  Knowledge Engineering , 1984, Annals of the New York Academy of Sciences.

[116]  Trevor J. M. Bench-Capon,et al.  Argument Based Machine Learning Applied to Law , 2005, Artificial Intelligence and Law.

[117]  Yoram Singer,et al.  A simple, fast, and effective rule learner , 1999, AAAI 1999.

[118]  Phan Minh Dung,et al.  On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games , 1995, Artif. Intell..

[119]  M. Pazzani Influence of prior knowledge on concept acquisition: Experimental and computational results. , 1991 .

[120]  Ivan Bratko,et al.  Argument-Based Machine Learning , 2006, ISMIS.

[121]  Laks V. S. Lakshmanan,et al.  Mining frequent itemsets with convertible constraints , 2001, Proceedings 17th International Conference on Data Engineering.

[122]  Nancy J. Cooke,et al.  Varieties of knowledge elicitation techniques , 1994, Int. J. Hum. Comput. Stud..

[123]  S. Gupta,et al.  Order Statistics from the Gamma Distribution , 1960 .

[124]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[125]  M. M. Groothuis,et al.  Expert system support and juridical quality , 2000 .

[126]  Ivan Bratko,et al.  Qualitatively Faithful Quantitative Prediction , 2003, IJCAI.

[127]  Luís Torgo,et al.  The Use of Domain Knowledge in Feature Construction for Financial Time Series Prediction , 2001, EPIA.

[128]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[129]  Yann LeCun,et al.  Tangent Prop - A Formalism for Specifying Selected Invariances in an Adaptive Network , 1991, NIPS.

[130]  Ivan Bratko,et al.  Argument Based Rule Learning , 2006, ECAI.

[131]  Carlos Iván Chesñevar,et al.  Integrating Defeasible Argumentation and Machine Learning Techniques , 2004, ArXiv.

[132]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[133]  Razvan C. Bunescu,et al.  Subsequence Kernels for Relation Extraction , 2005, NIPS.

[134]  G. Brier VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[135]  Ralph Grishman,et al.  Discovering Relations among Named Entities from Large Corpora , 2004, ACL.

[136]  Saso Dzeroski,et al.  Using Domain Knowledge on Population Dynamics Modeling for Equation Discovery , 2001, ECML.