From short-time diffusive to long-time ballistic dynamics: The unusual center-of-mass motion of quantum bright solitons