Vision for robotic object manipulation in domestic settings

In this paper, we present a vision system for robotic object manipulation tasks in natural, domestic environments. Given complex fetch-and-carry robot tasks, the issues related to the whole detect-approach-grasp loop are considered. Our vision system integrates a number of algorithms using monocular and binocular cues to achieve robustness in realistic settings. The cues are considered and used in connection to both foveal and peripheral vision to provide depth information, segmentation of the object(s) of interest, object recognition, tracking and pose estimation. One important property of the system is that the step from object recognition to pose estimation is completely automatic combining both appearance and geometric models. Experimental evaluation is performed in a realistic indoor environment with occlusions, clutter, changing lighting and background conditions.

[1]  Selim Benhimane,et al.  Vision-based control with respect to planar and non-planar objects using a zooming camera , 2003 .

[2]  Mårten Björkman.  Real-Time Motion and Stereo Cues for Active Visual Observers , 2002 .

[3]  Dana H. Ballard,et al.  Animate Vision , 1991, Artif. Intell..

[4]  Danica Kragic,et al.  A Framework for Visual Servoing , 2003, ICVS.

[5]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[6]  Jan-Olof Eklundh,et al.  Attending, Foveating and Recognizing Objects in Real World Scenes , 2004, BMVC.

[7]  Danica Kragic,et al.  Weak models and cue integration for real-time tracking , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[8]  Danica Kragic,et al.  Combination of foveal and peripheral vision for object recognition and pose estimation , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[9]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[10]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[11]  Steve Vinoski,et al.  CORBA: integrating diverse applications within distributed heterogeneous environments , 1997, IEEE Commun. Mag..

[12]  John Krumm,et al.  Object recognition with color cooccurrence histograms , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[13]  Cordelia Schmid,et al.  Indexing Based on Scale Invariant Interest Points , 2001, ICCV.

[14]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[15]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[17]  In-So Kweon,et al.  Robust model-based 3D object recognition by combining feature matching with tracking , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[18]  Roberto Cipolla,et al.  212D Visual Servoing with Respect to Planar Contours having Complex and Unknown Shapes , 2003, Int. J. Robotics Res..

[19]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[20]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[21]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[22]  Danica Kragic,et al.  Confluence of parameters in model based tracking , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[23]  Danica Kragic,et al.  Object recognition and pose estimation for robotic manipulation using color cooccurrence histograms , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[24]  Jan-Olof Eklundh,et al.  Real-Time Epipolar Geometry Estimation of Binocular Stereo Heads , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Danica Kragic.  Visual Servoing for Manipulation : Robustness and Integration Issues , 2001 .

[26]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[27]  Danica Kragic,et al.  Real-time tracking meets online grasp planning , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[28]  Jan-Olof Eklundh,et al.  A pure learning approach to background-invariant object recognition using pedagogical support vector learning , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.