System Support for Distributed Supercomputing on a Network of Workstations (Now)

Abstract : The goal of the NOW project was to explore and demonstrate a fundamental change in approach to the design and construction of large scale computing systems. This was motivated by the desire to deploy powerful systems very rapidly and to scale them incrementally, as is required to fully utilize commercial technologies that are advancing at a high rate, to meet new service demands that are increasing on internet time, and to address emergency or military situations. The key enabling technology for the project was the emergence of scalable, low latency, high bandwidth VLSI switches, pioneered in massively parallel processors and transferred into system area network (SAN) configurations. With SAN technology, it became feasible to construct powerful, integrated systems by literally plugging together many state of the art commercial workstations or PCs to form a high performance cluster. The project demonstrated the design approach, the solution to core challenges, and novel design opportunities by building and utilizing a cluster of over one hundred Ultrasparc workstations interconnected by a multigigabyte Myricom network.