Any production system intending to be competitive by means of reducing production costs and improving product quality which has to consider the use of automatic techniques. In order to promote robotic agricultural system, it is essential to develop the machine vision system. Over the last years there has been wisely used with CCD or CMOS camera; however, two main drawbacks of such system are: (1) depth information indirectly acquired and (2) being sensitive to various lighting condition. A novel 3D vision system was developed to address these issues, which used a laser range finder moving on a linear motion unit horizontally for collecting distance information around it. Some parameters potentially affecting the sensing performance were investigated. Depth image was generated after the system scanned for parts of fruit tree, an image processing algorithm combined scene knowledge (such as expected fruit size and the distance to the fruits) with shape features was put forward to recognize fruit. The experiment results show that the vision system with its capability of recognizing fruits under various conditions will enhance the overall performance of robotic fruit harvesting.