NIOSIT: efficient data access for log-structured merge-tree style storage systems

Recent years, the log-structured merge-tree(LSM-tree) style storage has been widely adopted in distributed data storage systems(e.g. Bigtable and HBase) and commercial database systems(e.g. Ocean-Base, Cassandra, SQLite, etc.) to provide both large-volume storage capacity and high-performance data updates. Write operations become easier as the LSM-tree style storage avoids writing in place by updating a data copy in memory. However, read operations are affected as it requires an additional step during a data compaction to check if there exists the newest update of data record in memory, which brings many of costly empty reads in real usage since the volume of immutable data is pervasively far more massive than incremental delta data. To address this issue, we design a new network request processing mechanism to allow data access being processed in an auxiliary lightweight network communication IO thread. And a Bloom filter is incorporated with the network IO thread to effectively filter out the empty reads. We also analyze the efficiency advantage of the mechanism and introduce its detailed implementation based on the well-known OceanBase system. Experimental study using the YCSB benchmark demonstrates the proposed mechanism can significantly achieve 20 percent to 30 percent better performance than existing method.