Stability of shock wave reflections in nonequilibrium steady flows and hysteresis