A prefetching prototype for the parallel file systems on the Paragon

Input-Output for parallel systems has drawn increasing attention in the last few years as it has become apparent that 1/0 performance rather than CPU or communication performance may be the limiting factor in future computing systems. A large number of applications in diverse areas such as large scale scientific computations, database and information processing, hypertext and multimedia systems, information retrieval etc. require processing very large quantities of data. For example, a typical Grand Challenge Application at present could require 1Gbyte to 4Tbytes of dat a per run [I]. These figures are expected to increase by orders of magnitude as teraflop machines make their appearance. Unfortunately, the performance of the 1/0 subsystems of MPPs has not kept pace with their processing and communication capabilities. A poor 1/0 capability can severely degrade the performance of an entire application. The subject of this paper is parallel file systems in parallel computers, and specifically read-ahead and prefetching. The goal of prefetching is essentially to access data from disks anticipating future use of the data. In a parallel file system a file is normally striped across a large number of 1/0 nodes and disks, and several processing nodes concurrently access the tiles. Thus, prefet thing strategies that may work reasonably well for sequential files in uniprocessor environments may not be extended in a straightforward manner because access patterns seen by the 1/0 nodes are interleaved accesses of many compute nodes. Accesses that exhibit good locality from a compute node’s perspective may not necessarily exhibit locality in a parallel striped file. In this paper, we describe the design and implementation of a prefetching prototype on the Paragon Parallel File System (PFS). It should be noted that the implementation of the prefetchlng strategy is done in a real and production parallel file system by modifying the code of the file system. Thus, all the overheads that come with the production software are also present in the prefetching software, which are normally ignored in simulations and other studies. The prototype has been extensively tested. In this paper, we present initial experience and performance studies from this