Towards heterogeneous integration of optical isolators and circulators with lasers on silicon [Invited]