A generalization of local class field theory by using $K$-groups, II