To high redshift and low mass: exploring the emergence of quenched galaxies and their environments at $3