Fourier transform infrared (FT-IR) spectroscopic imaging combines the specificity of optical microscopy with the spectral selectivity of vibrational spectroscopy. There is increasing recognition that the recorded data may be dependent on the optical configuration and sample morphology in addition to its local material spectral response, but a quantitative framework for predicting such dependence is lacking. Here, a theory is developed to relate recorded data to the spectral and physical properties of heterogeneous samples. The modeling approach combines optical theory through rigorous coupled wave analysis with modeling of sampling geometry and sample structure. The interplay of morphology and dispersion are systematically explored using increasingly sophisticated samples to illustrate the dependence of the detected optical intensity on the spatial sample structure. Predictions of spectral distortions arising from the sample structure are quantified, and experimental validation of the developed theory is performed using a microfabricated standard from a commercial photoresist polymer. The developed framework forms a basis for understanding sample induced distortions in spectroscopic IR microscopy and imaging.