Response of strongly interacting matter to a magnetic field: Some exact results