Sorting Large Files on a Backend Multiprocessor

The authors investigate the feasibility and efficiency of a parallel sort-merge algorithm by considering its implementation of the JASMIN prototype, a backend multiprocessor built around a fast packet bus. They describe the design and implementation of a parallel sort utility and present and analyze the results of measurements corresponding to a range of file sizes and processor configurations. The results show that using current, off-the-shelf technology coupled with a streamlined distributed operating system, three- and five-microprocessor configurations, provide a very cost-effective sort of large files. The three-processor configuration sorts a 100-Mb file in 1 hr which compares well to commercial sort packages available on high-performance mainframes. In additional experiments, the authors investigate a model to tune their sort software and scale their results to higher processor and network capabilities. >

[1]  Shimon Even,et al.  Parallelism in tape-sorting , 1974, Commun. ACM.

[2]  Forest Baskett,et al.  Task communication in DEMOS , 1977, SOSP '77.

[3]  Patricia G. Selinger,et al.  Access path selection in a relational database management system , 1979, SIGMOD '79.

[4]  Dina Bitton Friedland Design, analysis, and implementation of parallel external sorting algorithms , 1981 .

[5]  David J. DeWitt,et al.  Duplicate record elimination in large data files , 1983, TODS.

[6]  Yuzuru Tanaka A Data-Stream Database Machine with Large Capacity , 1983, Advanced Database Machine Architecture.

[7]  Hans Christoph Zeidler,et al.  RDBM: A Dedicated Multiprocessor System for Database Management , 1983, Advanced Database Machine Architecture.

[8]  Sudhur Ahuja S/NET: A High-Speed Interconnect for Multiple Computers , 1983, IEEE J. Sel. Areas Commun..

[9]  Kevin Wilkinson,et al.  Overview of the Jasmin database machine , 1984, SIGMOD '84.

[10]  J. P. Linderman The UNIX system: Theory and practice in the construction of a working sort routine , 1984, AT&T Bell Laboratories Technical Journal.

[11]  Michael Stonebraker,et al.  A measure of transaction processing power , 1985 .

[12]  Hiroshi Sakai,et al.  A hardware pipeline algorithm for relational database operation , 1985, ISCA '85.

[13]  Sai Choi Kwan External sorting: I/O analysis and parallel processing techniques , 1986 .

[14]  M. Kitsuregawa,et al.  Architecture and performance of relational algebra machine GRACE , 1989 .