论文信息 - Sorting on a Cluster Attached to a Storage-Area Network

Sorting on a Cluster Attached to a Storage-Area Network

In November 2004, the SAN Cluster Sort program (SCS) set new records for the Indy versions of the Minute and TeraByte Sorts. SCS ran on a cluster of 40 dual-processor Itanium2 nodes on the show floor at the Supercomputing 2004 conference (SC04), performing its data accesses to 240 SAN-attached 8+P RAID5 arrays managed by the IBM General Parallel File System. This hardware and software combination achieved peak data transfer rates of over 14GB/sec, while sorting a 125GB input file in 58.7 seconds, and a 1TB input file in 7 minutes, 17 seconds.

Jim Wyllie | J. Wyllie

[1] Frank B. Schmuck,et al. GPFS: A Shared-Disk File System for Large Computing Clusters , 2002, FAST.

[2] Jack Dongarra,et al. MPI: The Complete Reference , 1996 .

[3] Andrea C. Arpaci-Dusseau,et al. High-performance sorting on networks of workstations , 1997, SIGMOD '97.

[4] Michael Stonebraker,et al. A measure of transaction processing power , 1985 .

[5] David B. Lomet,et al. Alphasort: A cache-sensitive parallel external sort , 1995, The VLDB Journal.

[6] Jim Gray,et al. A Minute with Nsort on a 32P NEC Windows Itanium2 Server , 2004 .

[7] Anthony Skjellum,et al. A High-Performance, Portable Implementation of the MPI Message Passing Interface Standard , 1996, Parallel Comput..

[8] Jim Wyllie,et al. SPsort: How to Sort a Terabyte Quickly , 1999 .

[9] Brian C. Forney,et al. Datamation 2001: A Sorting Odyssey , 2001 .

[10] Alex Tsukerman,et al. FASTSORT: AN EXTERNAL SORT USING PARALLEL PROCESSING , 2002 .

[11] Ramesh C. Agarwal,et al. A super scalar sort algorithm for RISC processors , 1996, SIGMOD '96.