Peculiarities of reflection and transmission matrices of a system with magneto-optical activity and asymmetric transmission