The theory of open localized quantum systems in representation of the effective Hamiltonian