Natural HPC substrate: Exploitation of mixed multicore CPU and GPUs

Recent GPU developments have attracted much interest in the HPC community. Since each GPU interface requires a dedicated host processor, the unused high performance non-GPU processors are simply wasted. GPUs are energy intensive and are more likely to fail than CPUs, we are interested in using all processors to a) boosting application performance, and b) defending GPU failures. This paper reports parallel computation experiments using a natural semantic multiplexing substrate; we call Deeply Decoupled Parallel Processing (D2P2). The idea is to apply statistic multiplexing on application's semantic network with application-defined data tuples. Tuple space parallel processing is a natural choice for applying statistic multiplexing on application semantic networks. We report up to 53% performance gain for CPU:GPU capability ratio of 1:5. For faster GPUs, CPUs are better used to prevent application halt when GPU fails. The D2P2 substrate allows fault tolerant parallel processing using heterogeneous processors.