Optimal Control-Aware Transmission for Mission-Critical M2M Communications Under Bandwidth Cost Constraints

In this paper, we consider a mission-critical control system, where a dynamic plant is monitored by a mobile device (MD), and the monitored signal is transmitted to a remote controller via heterogeneous cellular and Wi-Fi networks. We propose an optimal control-aware machine-to-machine (M2M) transmission strategy for mission-critical control applications, in which control performance is measured by remote estimation error and system stability while limited by bandwidth cost. Specifically, the problem of minimizing estimation error, subject to the constraints of cellular usage costs and system stability, is formulated as an infinite-horizon constrained Markov decision process (CMDP), where the MD has options to transmit through Wi-Fi or cellular, or to stay idle. We solve the problem by utilizing the Lagrange multiplier approach, and prove that the optimal strategy is a randomized mixture of two threshold structure strategies. Furthermore, to estimate the structured optimal strategy, we present an algorithm called simultaneous perturbation stochastic approximation (SPSA), in which the complexity is <inline-formula> <tex-math notation="LaTeX">$O(|\mathcal {A}|)$ </tex-math></inline-formula> lower than a non-structured one with <inline-formula> <tex-math notation="LaTeX">$|\mathcal {A}|$ </tex-math></inline-formula> being the number of the actions.