Quantum phase transition in symmetric quantum three-body system