Symmetry and inverse-closedness of matrix algebras and functional calculus for infinite matrices