Network intrusion detection system based on protocol behavior