A Behind-the-Scenes Story on Applying Cross-Layer Coordination to Disks and RAIDs

Coordinating storage components across abstraction layers has demonstrated significant performance gains. However, when applied near the physical storage, this approach relies on exposing and exploiting low-level hardware characteristics, perhaps a large number of them, to cope with complex modern disks and RAIDs to apply such an approach. Through clean-room implementations and validations of prior research on track-aligned accesses and its incorporation in RAIDs, as well as through experiments with our proposed queue coordination in RAIDs, we confirmed that cross-layer coordination can indeed yield high performance gains. On the other hand, the effective use of cross-layer coordination involves overcoming several challenges: (1) developing efficient and automated ways to extract and exploit hardware characteristics due to rapidly evolving disks, (2) fostering a greater understanding of the legacy storage data path, so that we can better predict the benefits of low-level optimizations and their intertwined interactions, and (3) inventing efficient and automated ways to tune the low-level parameters.