Architectural interactions of i/o networks and inter-networks

Recent industrial efforts into next-generation System Area Networks (SAN) architectures have introduced new communication abstractions and protocols coupled with significant hardware support to meet performance goals of low latency, low host processor overhead and high throughput. They largely abandon existing IP-based network architectures under the perception that they cannot deliver sufficient performance. However, the success and ubiquity of TCP/IP based networks cannot be ignored. The suite of Internet protocols is well understood, offers a rich feature set and has been tested on a variety of platforms and links. This thesis examines the potential architectural interactions between new I/O networks, like Infiniband, and IP-based networks in the SAN space. We propose a hybrid of the two regimes called Queue Pair-IP (QPIP) that combines the Infiniband Queue Pair application abstraction layered over embedded IP-based protocols. Our hypothesis is that the Queue Pair enables effective offload of IP communication onto the network interface. To evaluate our hypothesis, we present the basis and design of the QPIP architecture. We analyze the individual components of the architecture, the QP and IP, and argue the case for their merger. The architecture seeks to exploit the synergy between the Infiniband QP and embedded IP to realize mutual benefit. The design itself is premised on providing memory-based messaging over an IP fabric. QPIP embraces simplicity and is engineered to be interoperable with other communication stacks. We demonstrate the effectiveness of QPIP through an implementation and empirical analysis. Using a programmable network adapter, we build a QPIP prototype and use it to conduct experiments in simple network benchmarks and a network storage application. The results show that QPIP, even with modest hardware support, provides low overhead, low latency and high throughput communication. We detail the lessons learned from the experiments and their impact on future network architectures.