Restrictions on realizable unitary operations imposed by symmetry and locality