Hadron yields and the phase diagram of strongly interacting matter