Berry phases for interacting spins in composite environments