A Light Touch Approach to Teaching Transformers Multi-view Geometry