The design of narrow-band Compton scattering sources for specific applications using nuclear resonance fluorescence (NRF) is presented. NRF lines are extremely narrow ($\ensuremath{\Delta}E/E\ensuremath{\sim}{10}^{\ensuremath{-}6}$) and require spectrally narrow sources to be excited selectively and efficiently. This paper focuses on the theory of spectral broadening mechanisms involved during Compton scattering of laser photons from relativistic electron beams. It is shown that in addition to the electron beam emittance, energy spread, and the laser parameters, nonlinear processes during the laser-electron interaction can have a detrimental effect on the gamma-ray source bandwidth, including a newly identified weakly nonlinear phase shift accumulated over the effective interaction duration. Finally, a design taking these mechanisms into consideration is outlined.