Using optical spectroscopy, we investigated the electrodynamic properties of Nb:SrTiO3/SrTiO3 superlattices. In these superlattices, a large enhancement of the Seebeck coefficient (S) has been reported with decreasing Nb:SrTiO3 layer thickness [refer to H. Ohta et al., Nature Mater. 6, 129 (2007)]. By analyzing the optical spectra, we found that the polaron plays an important role in determining the electrodynamic properties of the superlattices. With decreasing Nb:SrTiO3 layer thickness from eleven to one unit cell, we observed a threefold enhancement of the polaron effective mass and relaxation time. Such increases were attributed to a dimensional crossover of polaron from 3D to quasi-2D. Moreover, the modified nature of the polaron at low dimensions enhanced the thermoelectric properties of the oxide superlattice, by simultaneously increasing S and preventing the decrease of carrier mobility. Our results indicate that strong electron-phonon coupling can provide an alternative pathway in searching efficient thermoelectric materials.