Orthogonal sets of normal or conjugate-normal matrices