A Frame-work assisting the Visually Impaired People: Common Object Detection and Pose Estimation in Surrounding Environment

Many assisting systems to detect interested-objects (or obstacles) for Visually Impaired People(VIPs) have been studied for a long time. However, the practical systems built-in the real environment are still very challenging due to generic object classes in highly cluttered scenes (or complicated background). In this paper, we propose an unique frame-work to detect and estimate the full model of the common objects in the daily living of the VIP’s activities. Not only resolving the question where is the object’s query, the proposed system provides its relevant information that is presented such as size, and safety direction for grasping on a flat surface. The pipelines combine a series of the point cloud representation, table plane detection, objects detection and the full model estimation via a robust estimator. In this frame-work, we tackle that recent advantages of deep learning (e.g., RCNN, YOLO) that could be an efficient way for the detection task, while the geometry-based approaches estimate full 3-D model. This scheme does not require separating (or segmenting) the interested objects from the background of the surrounding scenes. The proposed system is compared with other approaches as well as is evaluated on the real datasets collected in common scenes such as Kitchen or cafeteria room. In these evaluations, the proposed frame-work meets requirements of high accuracy, processing time, and suitability for VIPs. The evaluation datasets are made publicity available.