In this paper, an approximate optimal control framework is developed to obtain a tracking controller for a nonlinear system that can be implemented as an online model-based learning approach. Assuming a structured unknown nonlinear system augmented with the dynamics of a commander system, we obtain a control rule minimizing a given quadratic tracking objective function. This is achieved by manipulating the cost and introducing a quadratic value function in terms of some nonlinear bases to comply with the structured dynamics. This problem formulation facilitates the computation of an update rule for the parameterized value function. As a result, a matrix differential equation of the coefficients is extracted, which gives a computationally efficient way for updating the value function and consequently attaining the tracking controller in terms of the reference and state trajectories. The proposed optimal tracking framework can be seen as an online model-based reinforcement learning approach, where we use a system identification method to update the system model, and generate a corresponding control in an iterative way. The presented learning algorithm is validated by implementing tracking control on two nonlinear benchmark problems.