Theta-Resonance: A Single-Step Reinforcement Learning Method for Design Space Exploration

—Given an environment (e.g., a simulator) for evaluating samples in a specified design space and a set of weighted eval- uation metrics—one can use θ -Resonance, a single-step Markov Decision Process (MDP), to train an intelligent agent producing progressively more optimal samples. In θ -Resonance, a neural network Net θ consumes a constant input tensor and produces a policy π θ as a set of conditional probability density functions (PDFs) for sampling each design dimension. We specialize existing policy gradient algorithms in deep reinforcement learning (D-RL) in order to use evaluation feedback (in terms of cost, penalty or reward) to update Net θ with robust algorithmic stability and minimal design evaluations. We study multiple neural architec- tures (for Net θ ) within the context of a simple SoC design space and propose a method of constructing synthetic space exploration problems to compare and improve design space exploration (DSE) algorithms. Although we only present categorical design spaces, we also outline how to use θ -Resonance in order to explore continuous and mixed continuous-discrete design spaces.