Going beyond the second virial coefficient in the hadron resonance gas model