In this paper, the mechanism of energy storage (ES)-based power oscillation damping is derived by the small signal and the classical electric torque method. And then, by cooperating PI with an integral reduction loop, a controller is designed to form a novel PI-IR controller to guarantee that the energy variation of ES damper is zero at the end of one oscillation. Furthermore, for the controller parameters tuning, the conventional model-based methods require a forecasting model on the uncertainty disturbances. To this end, this problem is formulated as a finite Markov decision process with unknown transition probability, and introduce a deep reinforcement learning (DRL) based model-free agent, the soft actor-critic, to obtain the real-time optimal control strategy. After numerous training, the well-trained agent can act as an experienced decision maker to provide the real-time near-optimal parameters setting for PI-IR control under different operating conditions. Time-domain and eigenvalue analysis results demonstrate the effectiveness of the proposed PI-IR controller and the superiority of the employed DRL based model-free method.