Using Kinect for monitoring warehouse order picking operations

In this paper we address the problem of monitoring warehouse order picking using a Kinect sensor, which provides RGB and depth information. We propose a new method that uses both 2D and 3D sensory data from the Kinect sensor for recognizing cuboids in an item picking scenario. 2D local texture based features are derived from the Kinect sensor’s RGB camera image data, which are used to distinguish objects with different patterns. 3D geometric information are derived from the Kinect sensor’s depth data, which are useful for recognizing objects of different size. Usually, 2D object recognition method has relatively low recognition accuracy when the object is not sufficiently textured or illuminated uniformly. Under those situations, 3D data provide geometric descriptions such as planes and volume and becomes a welcome addition to the 2D method. The proposed approach is implemented and tested on a simulated warehouse item picking workstation for item recognition and process monitoring. Many box-shape items of different sizes, shapes and pattern textures are tested. The proposed approach can also be applied in many other applications.