Diagrammatic many-body perturbation theory for general model spaces