Markov model prediction of I/O requests for scientific applications

Given the increasing performance disparity between processors and storage devices, exploiting knowledge of spatial and temporal I/O requests is critical to achieving high performance, particularly on parallel systems. Although perfect foreknowledge of I/O requests is rarely possible, even estimates of request patterns can potentially yield large performance gains. This paper evaluates Markov models to represent the spatial patterns of I/O requests in scientific codes. The paper also proposes three algorithms for I/O prefetching. Evaluation using I/O traces from scientific codes shows that highly accurate prediction of spatial access patterns, resulting in reduced execution times, is possible.

[1]  Daniel A. Reed,et al.  ARIMA time series modeling and forecasting for adaptive I/O prefetching , 2001, ICS '01.

[2]  Evgenia Smirni,et al.  Workload Characterization of Input/Output Intensive Parallel Applications , 1997, Computer Performance Evaluation.

[3]  Ray Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[4]  P. Krishnan,et al.  Optimal prefetching via data compression , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[5]  Daniel A. Reed,et al.  Automatic classification of input/output access patterns , 1997 .

[6]  Andrew A. Chien,et al.  Input/Output Characteristics of Scalable Parallel Applications , 1995, SC.

[7]  Andrew A. Chien,et al.  Performance Modeling of a Parallel I/O System: An Application Driven Approach , 1997, PPSC.

[8]  Brian N. Bershad,et al.  A trace-driven comparison of algorithms for parallel prefetching and caching , 1996, OSDI '96.

[9]  Daniel A. Reed,et al.  Input/output access pattern classification using hidden Markov models , 1997, IOPADS '97.

[10]  John Shalf,et al.  Diving deep: data-management and visualization strategies for adaptive mesh refinement simulations , 1999, Comput. Sci. Eng..

[11]  Edward Seidel,et al.  Three-dimensional numerical relativity with a hyperbolic formulation , 1998 .

[12]  Jesús Labarta,et al.  Linear aggressive prefetching: a way to increase the performance of cooperative caches , 1999, Proceedings 13th International Parallel Processing Symposium and 10th Symposium on Parallel and Distributed Processing. IPPS/SPDP 1999.

[13]  Ann Chervenak,et al.  Prefetching Without Hints: A Cost-Benefit Analysis for Predicted Accesses , 1999 .

[14]  Daniel A. Reed.  Scalable Input/Output: Achieving System Balance , 2003 .

[15]  Daniel A. Reed,et al.  Automatic arima time series modeling and forecasting for adaptive input/output prefetching , 2002 .

[16]  Andrew A. Chien,et al.  I/O requirements of scientific applications: an evolutionary view , 1996, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing.

[17]  D. Reed,et al.  Title: \lessons from Characterizing Input/output Bahavior of Parallel Scientiic Applications" Lessons from Characterizing Input/output Bahavior of Parallel Scientiic Applications , 2007 .

[18]  Daniel A. Reed,et al.  A Comparison of Logical and Physical Parallel I/o pAtterns , 1998, Int. J. High Perform. Comput. Appl..

[19]  P. Krishnan,et al.  Practical prefetching via data compression , 1993 .

[20]  Jim Zelenka,et al.  Informed prefetching and caching , 1995, SOSP.