LASSO: Location Assistant for Seeking and Searching Objects

Applying computer vision systems in robotic manufacturing to locate objects in various positions is an important approach in future manufacturing automation, e.g., product assembly tasks [1] and kitting tasks—grouping several parts to a container. However, the robot’s work environment is often large and cluttered with many objects, and this poses several challenges to computer vision systems. First, the accuracy of computer vision algorithms may be affected by cluttered backgrounds. Second, it is difficult to identify the target object when multiple objects are identical to the target. Third, using a single camera is not enough to cover all areas. To address the above challenges, we propose a programming system called LASSO (Location Assistant for Seeking and Searching Objects), which incorporates user hints to enhance multi-camera visual perception systems of robots in unstructured and cluttered environments. LASSO provides simple language for users to provide relative spatial relations as hints to narrow down the target object in its 3D space. LASSO then leverages this reduced search space to locate the target object in each camera view and compute the 3D location of the target object. A key feature of the LASSO system is that it quantifies the uncertainties of the 3D location, which provides users the quality of LASSO outputs. We demonstrate LASSO in both simulated and real world environments.