Detecting Anomalies and Intruders

Brittleness is a well-known problem in expert systems where a conclusion can be made, which human common sense would recognise as impossible e.g. that a male is pregnant. We have extended previous work on prudent expert systems to enable an expert system to recognise when a case is outside its range of experience. We have also used the same technique to detect new patterns of network traffic, suggesting a possible attack. In essence we use Ripple Down Rules to partition a domain, and add new partitions as new situations are identified. Within each supposedly homogeneous partition we use fairly simple statistical techniques to identify anomalous data. The special feature of these statistics is that they are reasonably robust with small amounts of data. This critical situation occurs whenever a new partition is added.