Stability and instability of Ellis and phantom wormholes: Are there ghosts?