Nonequilibrium critical scaling in quantum thermodynamics