Visual Interactions with Deep Models through Collaborative Semantic Inference