$H$ dibaryon away from the $SU(3)_f$ symmetric point