Eden: Simplified Management of Atypical High-Performance Computing Jobs

As multiprocessor and multicore technology becomes prevalent, shared-memory architectures with 1,024 or more processing cores are becoming available for general-purpose applications. As an early operator of such a system, the Remote Data Analysis and Visualization (RDAV) center at the University of Tennessee observed a host of user applications needing to scale up their computation by running many concurrent instances of generic codes. This isn't a typical way of using high-performance computing systems, and naive solutions supporting such needs would cause significant issues that hamper system scalability and stability. The RDAV center's Eden software package helps manage large numbers of concurrent serial jobs with high throughput for any such application. Here, the authors describe the motivation and technical nature of Eden and report representative use cases they've participated in during the past two years.