In the past few years, surface integral equation based domain decomposition method (SIE-DDM) has emerged as one of the most attractive approaches for modeling multiscale problems, which splits a large and complicated structure into a set of smaller and easier solvable sub-domains according to their geometrical properties. Some appropriate transmission conditions are employed to ensure the continuity of electric and magnetic fields between adjacent sub-domains. In the convention framework of SIE-DDM[1], an inner-outer iterative method is utilized to solve the final matrix equation. Essentially, this kind of iterative method can be viewed as a multiple right hand sides (MRHS) problem. However, if one sub-domain encounters the convergence problem, it is prohibitive for the whole solve process. Therefore, there are two particular motivations to develop a fast direct method as a sub-solver for SIE-DDM: 1)It is particularly efficient for MRHS problems. Iterative solution of SIE-DDM can be viewed as a situations involving MRHS problems. Matrices inversion for each sub-domain just need to be performed once by the present algorithm. During the outer iteration, applying its inverse to each additional updated right-hand side is inexpensive. 2)It is suitable for problems involving relatively ill-conditioned matrices. Although SIE-DDM decomposes the original large problem into smaller ones. It does not guarantee that every subdomain can be solved by iterative methods successfully at any time.