Software defined just-in-time caching in an enterprise storage system

A software defined storage environment is one in which logical storage resources and services are completely abstracted from physical storage systems. Therefore, not only can storage resources cross physical boundaries, but they can also be defined by software and provisioned automatically, for instance, by the applications that consume them. In this paper, we present a novel software defined cooperative caching (SDCC) framework that operates at the block layer and manages the placement of data in different tiers and caches that span multiple servers and storage systems in an integrated and coherent fashion. A programming interface complements the core framework by giving the applications an interface to control data organization across the storage, thereby allowing the block storage infrastructure to be software defined. The SDCC framework allows applications to actively influence the data layout while also benefitting from the system-wide knowledge and resource management capabilities of the storage system. We present an experimental study conducted using real workloads, and the results demonstrate the performance benefits gained with SDCC, as well as the potential for consolidating multiple different workloads that share the same storage server.