The dephasing of particle plasmons is investigated using light-scattering spectroscopy on individual gold nanoparticles. We find a drastic reduction of the plasmon dephasing rate in nanorods as compared to small nanospheres due to a suppression of interband damping. The rods studied here also show very little radiation damping, due to their small volumes. These findings imply large local-field enhancement factors and relatively high light-scattering efficiencies, making metal nanorods extremely interesting for optical applications. Comparison with theory shows that pure dephasing and interface damping give negligible contributions to the total plasmon dephasing rate.