Linux® Scalability for Large NUMA Systems

The SGI® Altix™ 3000 family of servers and superclusters are nonuniform memory access systems that support up to 64 Intel® Itanium® 2 processors and 512GB of main memory in a single Linux image. Altix is targeted to the high-performance computing (HPC) application domain. While this simplifies certain aspects of Linux scalability to such large processor counts, some unique problems have been overcome to reach the target of nearlinear scalability of this system for HPC applications. In this paper we discuss the changes that were made to Linux® 2.4.19 during the porting process and the scalability issues that were encountered. In addition, we discuss our approach to scaling Linux to more than 64 processors and we describe the challenges that remain in that arena.

[1]  Ray Bryant,et al.  Filesystem Performance and Scalability in Linux 2.4.17 , 2002, USENIX Annual Technical Conference, FREENIX Track.

[2]  Ray Bryant,et al.  PenguinOMeter: A New File-I/O Benchmark for Linux , 2001, Annual Linux Showcase & Conference.

[3]  John Hawkes.  Lockmeter: Highly Informative Instrumentation for Spin Locks in the Linux Kernel , 2000, Annual Linux Showcase & Conference.

[4]  D. Lenoski,et al.  The SGI Origin: A ccnuma Highly Scalable Server , 1997, Conference Proceedings. The 24th Annual International Symposium on Computer Architecture.

[5]  Anoop Gupta,et al.  The DASH Prototype: Logic Overhead and Performance , 1993, IEEE Trans. Parallel Distributed Syst..

[6]  R. Bryant.  Java Technology, Threads, and Scheduling in Linux , 2000 .

[7]  David Mosberger,et al.  IA-64 Linux Kernel: Design and Implementation , 2002 .