This paper proposes a novel three-dimensional trajectory tracking control methodology for a heterogeneous X-rudder autonomous underwater vehicle (XAUV) that can achieve finite-time convergence, complex actuator dynamics handling, and energy-efficient optimized rudder allocation. Under a compound robust control scheme, the trajectory tracking problem is decomposed into three sub-problems: kinematics control, dynamics control, and rudder allocation. For kinematics control, a novel finite-time line-of-sight (FTLOS) guidance law is proposed, which can achieve faster position and orientation tracking when compared with classical LOS guidance, and is rarely studied in the existing finite time control methods. In the dynamics control loop, global finite-time terminal sliding mode control (FTTSMC) laws are provided to solve the heading control, pitching control, and surge velocity tracking control problems, where finite-time convergence is achieved in both the approaching stage and sliding mode holding stage. The multi-source uncertainties with unknown upper bounds in both kinematics and dynamics loops are well treated by finite-time extended disturbance observers (FTEDOs), thus ensuring the system robustness. Moreover, the influence of complex actuator dynamics is fully considered by employing a RBFNN compensator to deal with the propeller saturation and proposing an energy-efficient optimal rudder allocator to tackle the multi-objective and multi-constraint heterogeneous X-rudder angle assignment problem. Finally, simulation verifications are carried out for two different scenarios, where Case 1 focuses on the adaptability of the algorithm to different conditions and Case 2 focuses on the superiority of the algorithm over three other commonly used algorithms. The comparative simulation results show that the proposed controller has good adaptability to different initial and disturbance conditions, and performs better than three other classical controllers, especially in convergence speed, tracking accuracy, stability, and energy consumption.