Can Evolutionary Computation Handle Large Datasets? A Study into Network Intrusion Detection

XCS is currently considered as the state of the art Evolutionary Learning Classifier Systems (ELCS). XCS has not been tested on large datasets, particularly in the intrusion detection domain. This work investigates the performance of XCS on the 1999 KDD Cup intrusion detection dataset, a real world dataset approximately five million records, more than 40 fields and multiple classes with non-uniform distribution. We propose several modifications to XCS to improve its detection accuracy. The overall accuracy becomes equivalent to that of traditional machine learning algorithms, with the additional advantages of being evolutionary and with O(n) complexity learner.