Beyond the block-based interface for flash-based storage

The block device interface has shaped the storage industry for almost three decades starting from the tape drives, hard-disks to cdroms. The block interface provides minimalist functionality of reading and writing and this simplicity and broad applicability of the block interface made it a standard. Unsurprisingly, the block interface has not fundamentally changed and the reasons for this goes back to the wide-spread adoption of the standard: the economic incentives not to change the interface outweighs the potential benefits. The approach of keeping the hardware interface minimal worked well for hard-disks: hard-disks are slow and implementing features in software was efficient. But, the arrival of newer media poses a challenge to this age-old wisdom. Flash can easily outperform traditional spinning disks, but until recently its exorbitant price made it uneconomical. With falling prices, triggered by the massive demand in consumer electronics, flash has suddenly become a very realistic replacement for hard-disks in the form of SSDs. Unfortunately, hiding the raw potential of flash behind a naive block interface is counter productive in more ways than one. Applications designed in a flash-oblivious manner perform tasks that adversely affect performance and device lifetime. Thus, the top heavy, software-only approach cannot be sustained. In this thesis, we investigate approaches to refactor the software. In the first part of the thesis, we explore transitioning a standard software feature, namely backups or snapshots, to the flash device. We design and implement ioSnap, which leverages flash to provide fast, file system independent and light-weight snapshots. Through ioSnap, we attempt to understand the tradeoffs of such a transition and the type of support necessary to make such a system feasible. In the second part of the thesis, we explore new interfaces to the flash device. New interfaces are necessary since we need a generic and systematic approach to expose data management abilities of flash. The new interfaces allow virtualization of address space and time by leveraging the characteristics of the underlying media. We demonstrate the usefulness of the new interfaces by making minimal changes to existing file-systems and applications to add features or simplify existing ones.