Virgo Cluster and field dwarf ellipticals in 3D – II. Internal dynamics points to tidal harassment?