CO-REDUCE: Collaborative Redundancy Reduction Service in Software-Defined Networks

A large portion of digital data is transferred repeatedly across networks and duplicated in storage systems, which costs excessive bandwidth, storage, energy, and operations. Thus, great effort has been made in both areas of networks and storage systems to lower the redundancies. However, due to the lack of the coordination capabilities, expensive procedures of C-H-I (Chunking, Hashing, and Indexing) are incurring recursively on the path of data processing. In this paper, we propose a collaborative redundancy reduction service (CO-REDUCE) in Software-Defined Networks (SDN). Taking advantage of SDN control, CO-REDUCE renders the promising vision of Redundancy Elimination as a network service (REaaS) as a real practical service. CO-REDUCE is a new virtualized network function service that dynamically offloads computational operations and memory management tasks of deduplication to the group of the software designed network middleboxes. Chaining various redundant REs of both storage and network into a service, COREDUCE consolidates and simplifies the expensive C-H-I processes. We develop service coordination protocols and virtualization and control mechanisms in SDN, and indexing algorithms for CO-REDUCE software-designed middleboxes (SDMB). Our evaluation results from the system and Mininet-based prototypes show that CO-REDUCE achieves 2-4 times more bandwidth reduction than existing RE technologies and has compatible storage space savings to existing storage de-duplication techniques while reducing expensive overhead of processing time and memory size.