Priority Neuron: A Resource-Aware Neural Network for Cyber-Physical Systems

Advances in sensing, computation, storage, and actuation technologies have entered cyber-physical systems (CPSs) into the smart era where complex control applications requiring high performance are supported. Neural networks (NNs) models are proposed as a predictive model to be used in model predictive control (MPC) applications. However, the ability to efficiently exploit resource hungry NNs in embedded resource-bound settings is a major challenge. In this paper, we propose priority neuron network (PNN), a resource-aware NNs model that can be reconfigured into smaller subnetworks at runtime. This approach enables a tradeoff between the model’s computation time and accuracy based on available resources. The PNN model is memory efficient since it stores only one set of parameters to account for various subnetwork sizes. We propose a training algorithm that applies regularization techniques to constrain the activation value of neurons and assigns a priority to each one. We consider the neuron’s ordinal number as our priority criteria in that the priority of the neuron is inversely proportional to its ordinal number in the layer. This imposes a relatively sorted order on the activation values. We conduct experiments to employ our PNN as the predictive model of a vehicle in MPC for path tracking. To corroborate the effectiveness of our proposed methodology, we compare it with two state-of-the-art methods for resource-aware NN design. Compared to state-of-the-art work, our approach can cut down the training time by 87% and reduce the memory storage by 75% while achieving similar accuracy. Moreover, we decrease the computation overhead for the model reduction process that searches for neurons below a threshold, from <inline-formula> <tex-math notation="LaTeX">$ {O(n)}$ </tex-math></inline-formula> to <inline-formula> <tex-math notation="LaTeX">$ {O}$ </tex-math></inline-formula>(log<inline-formula> <tex-math notation="LaTeX">$ {n}$ </tex-math></inline-formula>).