ExaPlan: Efficient Queueing-Based Data Placement, Provisioning, and Load Balancing for Large Tiered Storage Systems

Multi-tiered storage, where each tier consists of one type of storage device (e.g., SSD, HDD, or disk arrays), is a commonly used approach to achieve both high performance and cost efficiency in large-scale systems that need to store data with vastly different access characteristics. By aligning the access characteristics of the data, either fixed-sized extents or variable-sized files, to the characteristics of the storage devices, a higher performance can be achieved for any given cost. This article presents ExaPlan, a method to determine both the data-to-tier assignment and the number of devices in each tier that minimize the system’s mean response time for a given budget and workload. In contrast to other methods that constrain or minimize the system load, ExaPlan directly minimizes the system’s mean response time estimated by a queueing model. Minimizing the mean response time is typically intractable as the resulting optimization problem is both nonconvex and combinatorial in nature. ExaPlan circumvents this intractability by introducing a parameterized data placement approach that makes it a highly scalable method that can be easily applied to exascale systems. Through experiments that use parameters from real-world storage systems, such as CERN and LOFAR, it is demonstrated that ExaPlan provides solutions that yield lower mean response times than previous works. It supports standalone SSDs and HDDs as well as disk arrays as storage tiers, and although it uses a static workload representation, we provide empirical evidence that underlying dynamic workloads have invariant properties that can be deemed static for the purpose of provisioning a storage system. ExaPlan is also effective as a load-balancing tool used for placing data across devices within a tier, resulting in an up to 3.6-fold reduction of response time compared with a traditional load-balancing algorithm, such as the Longest Processing Time heuristic.

[1]  Christos Faloutsos,et al.  Using Utility to Provision Storage Systems , 2008, FAST.

[2]  Arif Merchant,et al.  Minerva: An automated resource provisioning tool for large-scale storage systems , 2001, TOCS.

[3]  Ahmed Amer,et al.  Predicting when not to predict , 2004, The IEEE Computer Society's 12th Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, 2004. (MASCOTS 2004). Proceedings..

[4]  Avani Wildani.  The Promise Of Data Grouping In Large Scale Storage Systems , 2013 .

[5]  Yusik Kim,et al.  ExaPlan: Queueing-Based Data Placement and Provisioning for Large Tiered Storage Systems , 2015, 2015 IEEE 23rd International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems.

[6]  Tayfur Altiok,et al.  Merging and splitting autocorrelated arrival processes and impact on queueing performance , 2008, Perform. Evaluation.

[7]  Joel L. Wolf,et al.  The placement optimization program: a practical solution to the disk file assignment problem , 1989, SIGMETRICS '89.

[8]  Anand Sivasubramaniam,et al.  HybridStore: A Cost-Efficient, High-Performance Storage System Combining SSDs and HDDs , 2011, 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.

[9]  Eric Anderson,et al.  Proceedings of the Fast 2002 Conference on File and Storage Technologies Hippodrome: Running Circles around Storage Administration , 2022 .

[10]  Himabindu Pucha,et al.  Cost Effective Storage using Extent Based Dynamic Tiering , 2011, FAST.

[11]  Chuan Heng Foh,et al.  Optimal disk storage allocation for multi-tier storage system , 2012, 2012 Digest APMRC.

[12]  Eric Anderson,et al.  Quickly finding near-optimal storage designs , 2005, TOCS.

[13]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[14]  Mark S. Squillante,et al.  Analysis and characterization of large‐scale Web server access patterns and performance , 1999, World Wide Web.

[15]  Ronald L. Graham,et al.  Bounds on Multiprocessing Timing Anomalies , 1969, SIAM Journal of Applied Mathematics.

[16]  S. Markoff,et al.  LOFAR - low frequency array , 2006 .

[17]  Ahmed Amer,et al.  Predictive data grouping: Defining the bounds of energy and latency reduction through predictive data grouping and replication , 2008, TOS.

[18]  Yifeng Zhu,et al.  Hot Random Off-Loading: A Hybrid Storage System with Dynamic Data Migration , 2011, 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.

[19]  John Frank Charles Kingman,et al.  The single server queue in heavy traffic , 1961, Mathematical Proceedings of the Cambridge Philosophical Society.

[20]  Antony I. T. Rowstron,et al.  Write off-loading: Practical power management for enterprise storage , 2008, TOS.