Practical machine learning based multimedia traffic classification for distributed QoS management

A multi-service Internet requires routers to recognise and prioritise IP flows carrying interactive or multimedia traffic. It is increasingly problematic for legal or administrative reasons to recognise such flows using unique port numbers or deep packet inspection. New work in recent years shows that Machine Learning (ML) techniques can use externally observable statistical characteristics to usefully differentiate such IP traffic. However, most previous work has not addressed the practicality of ML-based traffic classification in terms of CPU and memory usage. Here we describe our design, implementation and performance evaluation of a distributed, ML-based traffic classification and control system for FreeBSD's IP Firewall (IPFW). On an Intel Core i7 2.8 GHz PC our system can classify up to 400 000 packets per second using only one core and our system scales well to up to 100 000 simultaneous flows. Also our implementation allows one classifier PC to control subsequent traffic shaping or blocking at multiple (potentially lower performance) routers or gateways distributed around the network.

[1]  Anthony Lauck,et al.  Hashed and hierarchical timing wheels: efficient data structures for implementing a timer facility , 1997, TNET.

[2]  Yanghee Choi,et al.  Internet traffic classification demystified: on the sources of the discriminative power , 2010, CoNEXT.

[3]  Marco Canini,et al.  Experience with high-speed automated application-identification for network-management , 2009, ANCS '09.

[4]  Sebastian Zander,et al.  Design of DIFFUSE v0.1 – DIstributed firewall and Flow-shaper using statistical evidence , 2010 .

[5]  Ian Witten,et al.  Data Mining , 2000 .

[6]  Sebastian Zander,et al.  A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification , 2006, CCRV.

[7]  Grenville J. Armitage,et al.  Training on multiple sub-flows to optimise the use of Machine Learning classifiers in real-world IP networks , 2006, Proceedings. 2006 31st IEEE Conference on Local Computer Networks.

[8]  Grenville J. Armitage,et al.  A survey of techniques for internet traffic classification using machine learning , 2008, IEEE Communications Surveys & Tutorials.

[9]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[10]  Yan Luo,et al.  Acceleration of decision tree searching for IP traffic classification , 2008, ANCS '08.

[11]  Sebastian Zander,et al.  Automated network games enhancement layer: a proposed architecture , 2006, NetGames '06.

[12]  Xiaohong Guan,et al.  Traffic Classification - Towards Accurate Real Time Network Applications , 2007, HCI.

[13]  Maya Gokhale,et al.  Real-Time Classification of Multimedia Traffic Using FPGA , 2010, 2010 International Conference on Field Programmable Logic and Applications.

[14]  Benoit Claise,et al.  Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information , 2008, RFC.

[15]  Shun-Zheng Yu,et al.  Machine Learned Real-Time Traffic Classifiers , 2008, 2008 Second International Symposium on Intelligent Information Technology Application.

[16]  S. Zander,et al.  An Architecture for Automated Network Control of QoS over Consumer Broadband Links , 2005, TENCON 2005 - 2005 IEEE Region 10 Conference.

[17]  Andrew W. Moore,et al.  A Machine Learning Approach for Efficient Traffic Classification , 2007, 2007 15th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems.