Tracking an autonomous underwater vehicle (AUV) has been regarded as one of the most key applications for Internet of Underwater Things (IoUT). However, the strong mobility of AUV as well as asynchronous clock, stratification effect, and high energy consumption of acoustic communication make it challenging to achieve such a task. To handle the above issues, this article develops a ubiquitous tracking scheme for AUV. The tracking scheme is divided into two stages, i.e.: 1) motion prediction and 2) persistent tracking. In the first stage, an unscented transform-based localization estimator is utilized by sensor nodes to acquire the initial position of AUV, through which a terminal sliding-mode velocity observer is designed to predict the mobility trajectory of AUV. With the predicted mobility trajectory, a minimum rigid-graph-based tracking strategy is developed in the second stage to enable ubiquitously tracking. For the designed tracking strategy, the posterior Cramer–Rao lower bound is selected as the benchmark to optimize the network topology, such that a minimum rigid graph can be generated to balance the tradeoff between tracking accuracy and energy consumption. Particularly, the duty-cycle mechanism and the unscented Kalman filtering are jointly adopted to prolong the network lifetime and improve the tracking accuracy. Finally, simulation and experimental results are presented to show the effectiveness of our approach.