Algebraic $K$-theory of poly-(finite or cyclic) groups