Thermoelectric materials have recently attracted a lot of attention due to their ability to convert waste heat into electricity. Based on the extensive research in this area, the nanostructuring approach has been viewed as an effective strategy for increasing thermoelectric performance. This approach focuses on the formation and growth of the superfine, pure and uniform grain size. Since the grain size has a strong influence on the thermal conductivity, this can be reduced by increasing the phonon scattering at grain boundaries and refining the grain sizes. Therefore, this review aims to discuss the mechanism of reduction in thermal conductivity in small-grain zinc oxide (ZnO) and the optimization techniques for obtaining ZnO nanoparticles with desirably low thermal conductivity and excellent thermoelectric performance.