The dynamical fate of self-gravitating disc fragments after tidal downsizing