Efficient and Interpretable Robot Manipulation with Graph Neural Networks