Triple stereo vision system for safety monitoring of human-robot collaboration in cellular manufacturing

In a close proximity of a human-robot collaboration production system, safety monitoring has a paramount importance to ensure the human operator is being well protected throughout the collaborative operation with the robot manipulator. Due to the requirement to allow overlapping of working envelopes between these two parties, physical separation or two-dimensional sensory system is not effective as the safety measure for the production system. In the early development, safety monitoring by stereo vision system with two cameras was introduced to track the human operator's motion throughout the operation. Camera is used to capture images for tracking of color areas on the human operator. The image coordinates by particle filter and human body model are combined to estimate the 3D positions for the human motion monitoring. However, several weaknesses were observed in this development. For instance, due to the fixed camera viewing direction, occlusion of the detecting areas can severely affect the effectiveness of the safety monitoring. Therefore, one additional camera is added into the system to produce three pairs of stereo vision to improve the robustness towards lost tracking and occlusion tolerance. Hand position tracking experiment is conducted to evaluate the performance of the 3D position estimation.