I ran four million probits last night: HPC clustering with ParallelKnoppix

A high-performance computing (HPC) cluster is a group of computers that are networked together, and which have a software environment that allows a given computational task to be split up into parts that are evaluated on more than one CPU simultaneously. This can lead to dramatic reductions in the time needed to complete computations, in comparison to running them on a single CPU. ParallelKnoppix1 (PK) is a bootable CD that allows users with average computing skills to create an HPC cluster in very little time: about 10 minutes. The computers used in a PK cluster may be heterogeneous, and the cluster is temporary, in the sense that nothing is installed on the computers that are used in the cluster; they are not altered in any way. Thus, for example, the computers in a university computer room that are used by students during the day can be converted into an HPC cluster for night-time research work, without affecting their use by students the next day. This note describes what PK is, how it works, and how to set it up and use it. It shows how a Monte Carlo study that involves 4,000,000 nonlinear optimizations may be completed in less than 8 hours on a PK cluster, when it would take roughly 6.25 days on a single computer. The intention is to make econometricians aware of the existence of the tool and to give basic information on how to use it, along with pointers to additional sources of information. Creel (2005) presents additional examples of parallel programs and results that may be of interest to econometricians. The example presented in this paper as well as the examples from Creel (2005) are all contained on the latest version of the CD.