Vision-based Augmented Reality for Guiding Assembly

We consider the problem of scene augmentation in the context of a human engaged in assembling an object from its components. In order to exploit the potential of augmented reality (AR) in this context, two main problems need to be considered: designing an effective augmentation scheme for information presentation/control, and providing accurate and fast sensing to determine the state of the assembly. We utilized concepts from robot assembly planning to develop a systematic framework for presenting augmentation stimuli for the assembly domain. An interactive augmentation design and control engine called AUDIT is described. To provide sensing, we utilized computer vision methods for assembly object recognition without special markers. Even though fiducials currently constitute the only feasible vision-based solution, occlusion by the manipulator as well as other assembly parts make the use of more general computer vision techniques desirable. Here, we investigate computer vision techniques with the goal of eventually substituting markers. Constraints from the domain of assembly, as well as transformation space search-based algorithms make the problem tractable.