Simulation-based inference of deep fields: galaxy population model and redshift distributions