Development and evaluation of a low-cost scalable architecture for network traffic capture and storage for 10Gbps networks

The last years have witnessed an undoubtedly explosion of the Internet users’ demands for bandwidth. To manage such new demand and, especially, provide the adequate quality of service, ISPs have understood the importance of accurately monitoring their traffic, investing a great deal of effort in terms of funds and time. Novel packet I/O engines allow capturing traffic at multi-10Gbps using only-software and commodity hardware systems. This is achieved thanks to the application of techniques such as batch processing. Nevertheless, such novel I/O engines focus on packet processing throughput while shifting to the background the ability of recording the incoming packets in non-volatile storage systems. In this work the storage capabilities of novel I/O engines will be evaluated, and a scalable solution to this problem will be developed. Moreover, the use of batch processing involves degradation in the timestamp accuracy, which may be relevant for monitoring purposes. Two different approaches are proposed in this work to mitigate such effect: a simple algorithm to distribute inter-batch time among the packets composing a batch, and a driver modification to poll NIC buffers avoiding batch processing. Experimental results, using both synthetic and real traffic, show that our proposals allow capturing accurately timestamped traffic for monitoring purposes at multi-10Gbps rates.