CortexSuite: A synthetic brain benchmark suite

These days, many traditional end-user applications are said to “run fast enough” on existing machines, so the search continues for novel applications that can leverage the new capabilities of our evolving hardware. Foremost of these potential applications are those that are clustered around information processing capabilities that humans have today but are lacking in computers. The fact that brains can perform these computations serves as an existence proof that these applications are realizable. At the same time, we often discover that the human nervous system, with its 80 billion neurons, on some metrics, is more powerful and energy-efficient than today's machines. Both of these aspects make this class of applications a desirable target for an architectural benchmark suite, because there is evidence that these applications are both useful and computationally challenging. This paper details CortexSuite, a Synthetic Brain Benchmark Suite, which seeks to capture this workload. We classify and identify benchmarks within CortexSuite by analogy to the human neural processing function. We use the major lobes of the cerebral cortex as a model for the organization and classification of data processing algorithms. To be clear, our goal is not to emulate the brain at the level of the neuron, but rather to collect together synthetic, man-made algorithms that have similar function and have met with success in the real world. We consulted six world-class machine learning and computer vision researchers, who collectively hold 83,091 citations across their distinct subareas, asking them to identify newly emerging computationally-intensive algorithms or applications that are going to have a large impact over the next ten years. This is coupled with datasets that reflect the philosophy of practical use algorithms and are coded in “clean C” so as to make them accessible, analyzable, and usable for parallel and approximate compiler and architecture researchers alike.