CareDedup: Cache-Aware Deduplication for Reading Performance Optimization in Primary Storage

Deduplication technology has been increasingly used to reduce the primary storage cost. In practice, it often causes additional on-disk fragmentation that impairs the reading performance. Existing deduplication algorithms mainly focus on the static data layout design so that the random I/O requests are largely avoided and the harmful effect can be alleviated. However, our trace-driven emulations show that, deduplication does not always impair the reading. It offers unique new opportunities for reading performance optimization by more possible cache hits. Motivated by this, we propose a novel cache-aware deduplication scheme CareDedup to well leverage the new opportunities. Based on a uniform locality assessment algorithm design, CareDedup selects the most profitable duplicated blocks to deduplicate for maximizing the reading performance. Our experimental evaluation using real-world traces shows that compared with the sequence-based deduplication algorithms, the duplicate elimination ratio and the reading performance (latency) can be both improved simultaneously. Given a desired duplicate elimination ratio, CareDedup can consistently outperforms sequence-based method by further reducing the reading latency by 2-5%.