SepStore: Data Storage Accelerator for Distributed File Systems by Separating Small Files from Large Files

Distributed file systems often rely on disk file systems for storing data on disks. Disk file systems can do a relative good performance on large files than small files as sequential access patterns often exhibit for large files. This paper improves the performance of data servers for distributed file systems by improving the performance for small files. A LSM structure based key-value store is used for storing the data for small files for transforming the random access to sequential access as well as reducing the metadata of disk file systems. The key-value store is also used as the index for accessing small files. Experimental results showed that our method could improve the throughput up to 78% as well as 37% improvement on IOPS.