RTP-miner: a real-time security framework for RTP fuzzing attacks

Real-time Transport Protocol (RTP) is a widely adopted standard for transmission of multimedia traffic in Internet telephony (commonly known as VoIP). Therefore, it is a hot potential target for imposters who can launch different types of Denial of Service (DoS) attacks to disrupt communication; resulting in not only substantive revenue loss to VoIP operators but also undermining the reliability of VoIP infrastructure. The major contribution of this paper is an online framework -- RTP-Miner -- that detects RTP fuzzing attacks in realtime; as a result, it is not possible to deny access to legitimate users. RTP-Miner can detect both header and payload fuzzing attacks. Fuzzing in the header of RTP packets is detected by combining well known distance measures with a decision tree based classifier. In comparison, payload fuzzing is detected through a novel Markov state space model at the receiver. We evaluate RTP-Miner on a realworld RTP traffic dataset. The results show that RTP-Miner detects fuzzing in RTP header with more than 98% accuracy and less than 0.1% false alarm rate even when only 3% fuzzing is introduced. For the same fuzzing rate, it detects payload fuzzing -- a significantly more challenging problem -- with more than 80% accuracy and less than 2% false alarm rate. RTP-Miner has low memory and processing overheads that makes it well suited for deployment in real world VoIP infrastructure.