An active visual estimator for dexterous manipulation

We present a working implementation of a dynamics based architecture for visual sensing. This architecture provides field rate estimates of the positions and velocities of two independent falling balls in the face of repeated visual occlusions and departures from the field of view. The practical success of this system can be attributed to the interconnection of two strongly nonlinear dynamical systems: a novel triangulating state estimator; and an image plane window controller. We detail the architecture of this active sensor, provide data documenting its performance, and offer an analysis of its soundness in the form of a convergence proof for the estimator and a boundedness proof for the controller.

[1]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[2]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[3]  P. Anandan.  Measuring Visual Motion From Image Sequences , 1987 .

[4]  Russell L. Anderson,et al.  A Robot Ping-Pong Player: Experiments in Real-Time Intelligent Control , 1988 .

[5]  Daniel E. Koditschek,et al.  A family of robot control strategies for intermittent dynamical environments , 1990, IEEE Control Systems Magazine.

[6]  Daniel E. Koditschek,et al.  A family of robot control strategies for intermittent dynamical environments , 1990 .

[7]  John M. Hollerbach,et al.  Autonomous Robot Calibration for Hand-Eye Coordination , 1991, Int. J. Robotics Res..

[8]  Daniel E. Koditschek,et al.  Preliminary Experiments in Spatial Robot Juggling , 1991, ISER.

[9]  Daniel E. Koditschek,et al.  Progress in spatial robot juggling , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[10]  Mrdjan Jankovic,et al.  Some Problems In Perspective System Theory And Its Application To Machine Vision , 1992, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Daniel E. Koditschek,et al.  Distributed real-time control of a spatial robot juggler , 1992, Computer.

[12]  Peter K. Allen,et al.  Trajectory filtering and prediction for automated tracking and grasping of a moving object , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[13]  Hugh F. Durrant-Whyte,et al.  A Fully Decentralized Multi-Sensor System For Tracking and Surveillance , 1993, Int. J. Robotics Res..

[14]  Peter I. Corke,et al.  Video-rate robot visual servoing , 1993 .

[15]  Peter K. Allen,et al.  Automated tracking and grasping of a moving object with a robotic hand-eye system , 1993, IEEE Trans. Robotics Autom..

[16]  Koichi Hashimoto,et al.  Visual Servoing: Real-Time Control of Robot Manipulators Based on Visual Sensory Feedback , 1993 .

[17]  S. Hutchinson,et al.  Exploiting visual constraints in the synthesis of uncertainty-tolerant motion plans. I. The directional backprojection , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[18]  Armando Fox,et al.  Exploiting visual constraints in the synthesis of uncertainty-tolerant motion plans. II. The nondirectional backprojection , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[19]  Nikolaos Papanikolopoulos,et al.  Adaptive robotic visual tracking: theory and experiments , 1993, IEEE Trans. Autom. Control..

[20]  Nikolaos Papanikolopoulos,et al.  Visual Servoing for Robotic Assembly , 1993 .

[21]  W. Stronge,et al.  Principles of Mechanics , 1993 .

[22]  Daniel E. Koditschek,et al.  Further progress in robot juggling: the spatial two-juggle , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[23]  Daniel E. Koditschek,et al.  Toward the control of attention in a dynamically dexterous robot , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[24]  Pradeep K. Khosla,et al.  Integrating Sensor Placement and Visual Tracking Strategies , 1993, ISER.

[25]  Gregory D. Hager,et al.  Robot feedback control based on stereo vision: towards calibration-free hand-eye coordination , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[26]  Gregory D. Hager,et al.  Robot hand-eye coordination based on stereo vision , 1995 .

[27]  B. Ghosh,et al.  Visually guided ranging from observations of points, lines and curves via an identifier based nonlinear observer , 1995 .

[28]  Gregory D. Hager,et al.  A "robust" convergent visual servoing system , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[29]  Daniel E. Koditschek,et al.  An active visual estimator for dexterous manipulation , 1996, IEEE Trans. Robotics Autom..

[30]  Pradeep K. Khosla,et al.  Robotic visual servoing and robotic assembly tasks , 1996, IEEE Robotics Autom. Mag..