Automatic sorting system for industrial robot with 3D visual perception and natural language interaction

With the development of the technologies such as computer vision, natural language interaction, process control technology, and sensor technology, the discussion of automatic sorting system of robot hits a hotspot in the robot community. This paper presents an automatic sorting system for industrial robot with the technologies of 3D visual perception, natural language interaction and automatic programming. Therein, a ‘rule-scene’ matching and interaction algorithm is proposed to combine all these modules together. By utilising our algorithm, robot can interact with human beings according to the real-time actual three-dimensional scene information and can guide users to give correct rules through speech when the rule is invalid. After getting the correct rules, robot can sort the object automatically using the automatic programming and execution algorithm designed in this study. In the experimental section, the designed system is applied to a fruit-sorting scene, which proves the effectiveness and practicability of the system.