Comprehensibility & Overfitting Avoidance in Genetic Programming for Technical Trading Rules

This paper presents two methods for increasing comprehensibility in technical trading rules produced by Genetic Programming. For this application domain adding a complexity penalizing factor to the objective fitness function also avoids overfitting the training data. Using pre-computed derived technical indicators, although it biases the search, can express complexity while retaining comprehensibility. Several of the learned technical trading rules outperform a buy and hold strategy for the S&P500 on the testing period from 1990-2002, even taking into account transaction costs.

[1]  Michael J. Pazzani,et al.  Beyond Concise and Colorful: Learning Intelligible Rules , 1997, KDD.

[2]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[3]  Paul R. Cohen,et al.  Multiple Comparisons in Induction Algorithms , 2000, Machine Learning.

[4]  David Haussler,et al.  Occam's Razor , 1987, Inf. Process. Lett..

[5]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[6]  Franklin Allen,et al.  Using genetic algorithms to find technical trading rules , 1999 .

[7]  Christopher J. Neely,et al.  Is Technical Analysis in the Foreign Exchange Market Profitable? A Genetic Programming Approach , 1996, Journal of Financial and Quantitative Analysis.

[8]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[9]  Pedro M. Domingos.  The Role of Occam's Razor in Knowledge Discovery , 1999, Data Mining and Knowledge Discovery.

[10]  Allen Newell,et al.  Chunking in Soar: The anatomy of a general learning mechanism , 1985, Machine Learning.

[11]  Shu-Heng Chen,et al.  Genetic Algorithms and Genetic Programming in Computational Finance , 2002 .

[12]  Alex A. Freitas,et al.  Data Mining with Constrained-syntax Genetic Programming: Applications in Medical Data Sets , 2001 .

[13]  B. A. Shepherd,et al.  An Appraisal of a Decision Tree Approach to Image Classification , 1983, IJCAI.

[14]  J. Ross Quinlan,et al.  Simplifying Decision Trees , 1987, Int. J. Man Mach. Stud..

[15]  John Mingers,et al.  An Empirical Comparison of Pruning Methods for Decision Tree Induction , 1989, Machine Learning.

[16]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[17]  Wray L. Buntine,et al.  A Further Comparison of Splitting Rules for Decision-Tree Induction , 1992, Machine Learning.

[18]  J. Ross Quinlan,et al.  Simplifying decision trees , 1987, Int. J. Hum. Comput. Stud..

[19]  Katia Sycara,et al.  The Importance of Simplicity and Validation in Genetic Programming for Data Mining in Financial Data , 1999 .