Global positioning system (GPS) has been widely used to estimate the absolute position of outdoor vehicles. Global positioning system and inertial measurement unit (IMU) can be integrated to deliver an enhanced effect over the individual system. GPS/IMU navigation solution that utilizes time-differenced carrier phase (TDCP) as observation can achieve high-precision positioning. The traditional TDCP-GPS/IMU navigation methods are compromised by the loss of constraints, especially when cycle slip occurs. This article proposes an optimal time difference-based TDCP-GPS/IMU navigation using graph optimization. Different from current methods, an optimal time difference strategy is designed to formulate integer-free constraints that span multiple epochs in a time window. Each satellite can be fully used to restrain the accumulation errors even though cycle slip happens. Then, the integer-free constraints are fused with relative constraints from IMU preintegration and TDCP between two successive epochs using graph optimization. Field tests are carried out to validate the performance of the proposed method. The results show that the proposed method can make the most of window length constraints to improve the positioning performance compared with traditional methods.