An In-Depth Analysis of Cloud Block Storage Workloads in Large-Scale Production

Cloud block storage systems support diverse types of applications in modern cloud services. Characterizing their I/O activities is critical for guiding better system designs and optimizations. In this paper, we present an in-depth analysis of production cloud block storage workloads through the block-level I/O traces of billions of I/O requests collected from Alibaba Cloud. We study the characteristics of load intensity, spatial patterns, and temporal patterns. Also, we present a comparative study on our traces and the notable public block-level I/O traces from Microsoft Research Cambridge, and identify the commonalities and differences of the two sets of traces. Finally, we provide 15 findings and discuss their implications on load balancing, cache efficiency, and storage cluster management in a cloud block storage system. Our traces are now released for public use.

[1]  Qiang Cao,et al.  BCW: Buffer-Controlled Writes to HDDs for SSD-HDD Hybrid Storage Server , 2020, FAST.

[2]  Antony I. T. Rowstron,et al.  Write off-loading: Practical power management for enterprise storage , 2008, TOS.

[3]  Mariko Sugawara,et al.  Understanding storage traffic characteristics on enterprise virtual desktop infrastructure , 2017, SYSTOR.

[4]  Andrew Warfield,et al.  Characterizing Storage Workloads with Counter Stacks , 2014, OSDI.

[5]  Akshat Verma,et al.  SRCMap: Energy Proportional Storage Using Dynamic Consolidation , 2010, FAST.

[6]  Qing Yang,et al.  WARCIP: write amplification reduction by clustering I/O pages , 2019, SYSTOR.

[7]  Erez Zadok,et al.  Distribution Fitting and Performance Modeling for Storage Traces , 2019, 2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS).

[8]  Chita R. Das,et al.  Towards characterizing cloud backend workloads: insights from Google compute clusters , 2010, PERV.

[9]  Tao Xie,et al.  I/O Characteristics of Smartphone Applications and Their Implications for eMMC Design , 2015, 2015 IEEE International Symposium on Workload Characterization.

[10]  Irfan Ahmad.  Easy and Efficient Disk I/O Workload Characterization in VMware ESX Server , 2007, 2007 IEEE 10th International Symposium on Workload Characterization.

[11]  Jiesheng Wu,et al.  Lessons and Actions: What We Learned from 10K SSD-Related Storage System Failures , 2019, USENIX Annual Technical Conference.

[12]  Kai Chen,et al.  URSA: Hybrid Block Storage for Cloud-Scale Virtual Disks , 2019, EuroSys.

[13]  Vagelis Hristidis,et al.  BORG: Block-reORGanization for Self-optimizing Storage Systems , 2009, FAST.

[14]  Sang-Won Lee,et al.  SFS: random write considered harmful in solid state drives , 2012, FAST.

[15]  Andrea C. Arpaci-Dusseau,et al.  Slacker: Fast Distribution with Lazy Docker Containers , 2016, FAST.

[16]  Qiao Li,et al.  Access Characteristic Guided Read and Write Cost Regulation for Performance Improvement on Flash Memory , 2016, FAST.

[17]  Dutch T. Meyer,et al.  Parallax: virtual disks for virtual machines , 2008, Eurosys '08.

[18]  Qiang Cao,et al.  Analysis of and Optimization for Write-dominated Hybrid Storage Nodes in Cloud , 2019, SoCC.

[19]  Onur Mutlu,et al.  Data retention in MLC NAND flash memory: Characterization, optimization, and recovery , 2015, 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA).

[20]  Andrea C. Arpaci-Dusseau,et al.  The Unwritten Contract of Solid State Drives , 2017, EuroSys.

[21]  Hamid Sarbazi-Azad,et al.  DiskAccel: Accelerating Disk-Based Experiments by Representative Sampling , 2015, SIGMETRICS.

[22]  Mendel Rosenblum,et al.  The design and implementation of a log-structured file system , 1991, SOSP '91.

[23]  Qi Zhang,et al.  Characterization of storage workload traces from production Windows Servers , 2008, 2008 IEEE International Symposium on Workload Characterization.

[24]  Irfan Ahmad,et al.  Efficient MRC Construction with SHARDS , 2015, FAST.

[25]  Tzi-cker Chiueh,et al.  Software Orchestrated Flash Array , 2014, SYSTOR 2014.

[26]  Michael M. Swift,et al.  FlashTier: a lightweight, consistent and durable storage cache , 2012, EuroSys '12.

[27]  Asim Kadav,et al.  Blizzard: Fast, Cloud-scale Block Storage for Cloud-oblivious Applications , 2014, NSDI.

[28]  Ke Zhou,et al.  Efficient SSD Caching by Avoiding Unnecessary Writes using Machine Learning , 2018, ICPP.

[29]  Patrick P. C. Lee,et al.  Parity logging with reserved space: towards efficient updates and recovery in erasure-coded clustered storage , 2014, FAST.

[30]  Yiming Zhang,et al.  PBS: An Efficient Erasure-Coded Block Storage System Based on Speculative Partial Writes , 2020, ACM Trans. Storage.

[31]  Wei Wu,et al.  Optimizing NAND flash-based SSDs via retention relaxation , 2012, FAST.

[32]  Mahesh Balakrishnan,et al.  Extending SSD Lifetimes with Disk-Based Write Caches , 2010, FAST.