Forward Modeling of Galaxy Populations for Cosmological Redshift Distribution Inference