Extracting information ASAP!

Designing I/O systems capable of scaling up to deal with the next generation of extreme scale scientific environments is a significant challenge. Scientific applications already strain the capabilities of current filesystems and storage systems. This work presents a middleware-based approach that generalizes from previous work on staging areas to focus more generally on staging resources. Exploiting the steady increase of the ratio of compute capability to I/O bandwidth, the EnStage middleware system allows for metadata characterization and I/O processing to occur when as where appropriate. This includes in reserved staging areas, buffered memory, and even in the writing processes' execution context. Using the EnStage extension to previous work, we find a 1.4% increase in runtime due to additional functionality resulted in I/O time in the staging area dropping to only 16% of the non-reduced output.