Biophysical processes supporting the diversity of microbial life in soil