WAN OPTIMIZATION BY APPLICATION INDEPENDENT DATA REDUNDANCY ELIMINATION (BY CACHING)

Huge amount of data flows through internet today, the data traffic over the internet is more than 25,000 GB per second. This leads to problems such as congestion, latency, low throughput etc. So for a smooth user experience the Wide Area Network (WAN) should be optimized. Many techniques called WAN optimization techniques are available in order to increase data-flow efficiencies across wan. Main wan optimization techniques include deduplication, caching, latency optimization, traffic shaping, protocol spoofing etc. This article mainly discusses the deduplication technique for wan optimization. Deduplication eliminates the transfer of redundant data across the network, it does so by sending a reference to data and not sending the data itself. Deduplication is also called as Data Redundancy Elimination or DRE. DRE can be implemented either at packet level where we search and eliminate redundant packets or at byte level where we search for repeated byte string and replace it with a shorter reference. Surveys have shown that the DRE has ability to reduce the data traffic by 15% to 60%. This article discusses major DRE techniques which are available and challenges for implementing DRE algorithms.