New variables for classical and quantum gravity in all dimensions: IV. Matter coupling