The Hilbert-Schmidt method in the three-particle problem and determination of the vertex parts tdn and td*n