Robotic Work-Space Sensing and Control

Industrial robots are fast and accurate when working with known objects at precise locations in well-structured manufacturing environments, as done in the classical automation setting. In one sense, limited use of sensors leaves robots blind and numb, unaware of what is happening in their surroundings. Whereas equipping a system with sensors has the potential to add new functionality and increase the set of uncertainties a robot can handle, it is not as simple as that. Often it is difficult to interpret the measurements and use them to draw necessary conclusions about the state of the work space. For effective sensor-based control, it is necessary to both understand the sensor data and to know how to act on it, giving the robot perception-action capabilities. This thesis presents research on how sensors and estimation techniques can be used in robot control. The suggested methods are theoretically analyzed and evaluated with a large focus on experimental verification in real-time settings. One application class treated is the ability to react fast and accurately to events detected by vision, which is demonstrated by the realization of a ball-catching robot. A new approach is proposed for performing high-speed color-based image analysis that is robust to varying illumination conditions and motion blur. Furthermore, a method for object tracking is presented along with a novel way of Kalman-filter initialization that can handle initial-state estimates with infinite variance. A second application class treated is robotic assembly using force control. A study of two assembly scenarios is presented, investigating the possibility of using force-controlled assembly in industrial robotics. Two new approaches for robotic contact-force estimation without any force sensor are presented and validated in assembly operations. The treated topics represent some of the challenges in sensor-based robot control, and it is demonstrated how they can be used to extend the functionality of industrial robots. (Less)

[1]  J. L. Harris,et al.  Image Evaluation and Restoration , 1966 .

[2]  M. Farooq,et al.  Information type filters for tracking a maneuvering target , 1990 .

[3]  Anders Robertsson,et al.  Robotic assembly of emergency stop buttons , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Peter Cheeseman,et al.  On the Representation and Estimation of Spatial Uncertainty , 1986 .

[5]  Di Xiao,et al.  Sensor-based hybrid position/force control of a robot manipulator in an uncalibrated environment , 2000, IEEE Trans. Control. Syst. Technol..

[6]  M. Spong,et al.  Robot Modeling and Control , 2005 .

[7]  S. F. Schmidt,et al.  Application of State-Space Methods to Navigation Problems , 1966 .

[8]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision: From Images to Geometric Models , 2003 .

[9]  Jian Sun,et al.  Drag-and-drop pasting , 2006, SIGGRAPH 2006.

[10]  Mario Sznaier,et al.  Dynamics Based Robust Motion Segmentation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  Haiyan Wu,et al.  Dynamic manipulation: Nonprehensile ball catching , 2010, 18th Mediterranean Conference on Control and Automation, MED'10.

[12]  L. B. Weiner.  Kalman Filter Initialization With Large Initial Uncertainty And Strong Measurement Nonlinearity , 1981 .

[13]  Lars Nielsen,et al.  Torque-limited path following by online trajectory time scaling , 1990, IEEE Trans. Robotics Autom..

[14]  M. Ibrahim Sezan,et al.  Survey of recent developments in digital image restoration. , 1990 .

[15]  Berthold Bäuml,et al.  Kinematically optimal catching a flying ball with a hand-arm-system , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Angel Valera,et al.  Force estimation and control in robot manipulators , 2003 .

[17]  R. Bellman.  The theory of dynamic programming , 1954 .

[18]  M. Steinbuch,et al.  Trajectory planning and feedforward design for high performance motion systems , 2004, Proceedings of the 2004 American Control Conference.

[19]  Hugh F. Durrant-Whyte,et al.  Simultaneous Localization and Mapping with Sparse Extended Information Filters , 2004, Int. J. Robotics Res..

[20]  John J. Craig,et al.  Hybrid position/force control of manipulators , 1981 .

[21]  Patrick Pérez,et al.  Color-Based Probabilistic Tracking , 2002, ECCV.

[22]  Randall Smith,et al.  Estimating Uncertain Spatial Relationships in Robotics , 1987, Autonomous Robot Vehicles.

[23]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[24]  Anders Robertsson,et al.  Flexible application development and high-performance motion control based on external sensing and reconfiguration of ABB industrial robot controllers , 2010, ICRA 2010.

[25]  Kaan Erkorkmaz.  Optimal trajectory generation and precision tracking control for multi-axis machines , 2004 .

[26]  H. Frank,et al.  Accurate prediction of interception positions for catching thrown objects in production systems , 2008, 2008 6th IEEE International Conference on Industrial Informatics.

[27]  Sonja Macfarlane,et al.  On-line smooth trajectory planning for manipulators , 2001 .

[28]  Carlos Canudas de Wit,et al.  Friction Models and Friction Compensation , 1998, Eur. J. Control.

[29]  Andrew A. Goldenberg,et al.  Modeling of friction using spectral analysis , 1998, IEEE Trans. Robotics Autom..

[30]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[31]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[32]  Anders Robertsson,et al.  Force controlled robotic assembly without a force sensor , 2012, 2012 IEEE International Conference on Robotics and Automation.

[33]  Feng Wu,et al.  Very Fast Template Matching , 2002, ECCV.

[34]  Peter C. Cheeseman,et al.  Estimating uncertain spatial relationships in robotics , 1986, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[35]  Feng Li,et al.  Blurred target tracking by Blur-driven Tracker , 2011, 2011 International Conference on Computer Vision.

[36]  Pradeep K. Khosla,et al.  Vision and force driven sensorimotor primitives for robotic assembly skills , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[37]  Helge J. Ritter,et al.  On-line planning of time-optimal, jerk-limited trajectories , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[39]  Anders Robertsson,et al.  Force/Vision Based Active Damping Control of Contact Transition in Dynamic Environments , 2006, WDV.

[40]  Gunnar Bolmsjö,et al.  Extending an industrial robot controller: implementation and applications of a fast open sensor interface , 2005, IEEE Robotics & Automation Magazine.

[41]  Christopher David Cook,et al.  Sensorless Force Estimation for Robots with Friction , 2002 .

[42]  N S Kopeika,et al.  Comparison of direct blind deconvolution methods for motion-blurred images. , 1999, Applied optics.

[43]  Hanqing Lu,et al.  Real-time visual tracking via Incremental Covariance Tensor Learning , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[44]  Karl Johan Åström,et al.  Object tracking with measurements from single or multiple cameras , 2010, 2010 IEEE International Conference on Robotics and Automation.

[45]  Satish S. Nair,et al.  Modeling and compensation of low-velocity friction with bounds , 1999, IEEE Trans. Control. Syst. Technol..

[46]  Alin Albu-Schäffer,et al.  The DLR lightweight robot: design and control concepts for robots in human environments , 2007, Ind. Robot.

[47]  Herman Bruyninckx,et al.  Hierarchical Dirichlet Process Hidden Markov Models for abnormality detection in robotic assembly , 2012 .

[48]  Günter Schreiber,et al.  Off-the-shelf vision for a robotic ball catcher , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[49]  Tao Ding,et al.  A rank minimization approach to fast dynamic event detection and track matching in video sequences , 2007, 2007 46th IEEE Conference on Decision and Control.

[50]  Matthew R. Walter,et al.  Exactly Sparse Extended Information Filters for Feature-based SLAM , 2007, Int. J. Robotics Res..

[51]  Jan M. Maciejowski,et al.  Predictive control : with constraints , 2002 .

[52]  J. De Schutter,et al.  A roadmap for autonomous robotic assembly , 2001, Proceedings of the 2001 IEEE International Symposium on Assembly and Task Planning (ISATP2001). Assembly and Disassembly in the Twenty-first Century. (Cat. No.01TH8560).

[53]  Hanzi Wang,et al.  Generalized Kernel-Based Visual Tracking , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[54]  Karl G. Kempf,et al.  Opportunistic scheduling for robotic assembly , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[55]  Oliver Birbach,et al.  Estimation and prediction of multiple flying balls using Probability Hypothesis Density filtering , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[56]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[57]  Vincenzo Lippiello,et al.  3D monocular robotic ball catching with an iterative trajectory estimation refinement , 2012, 2012 IEEE International Conference on Robotics and Automation.

[58]  Christian Balkenius,et al.  Color constancy in visual scene perception , 2003 .

[59]  E H Land,et al.  Recent advances in retinex theory and some implications for cortical computations: color vision and the natural image. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[60]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[61]  Raja Chatila,et al.  An Experimental System for Incremental Environment Modelling by an Autonomous Mobile Robot , 1989, ISER.

[62]  Matthew T. Mason,et al.  Compliance and Force Control for Computer Controlled Manipulators , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[63]  Alexander Verl,et al.  A Framework for a Fault Tolerant and Learning Robotic Assembly System , 2012, ROBOTIK.

[64]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[65]  Alexander Dietrich,et al.  Catching flying balls and preparing coffee: Humanoid Rollin'Justin performs dynamic and sensitive tasks , 2011, 2011 IEEE International Conference on Robotics and Automation.

[66]  Won Hong,et al.  Robotic catching and manipulation using active vision , 1995 .

[67]  Paolo Rocco,et al.  Implicit force control for industrial robots in contact with stiff surfaces , 1996, Autom..

[68]  Horst-Michael Groß,et al.  A Hybrid Kalman Filter Based Algorithm for Real-time Visual Obstacle Detection , 2007, EMCR.

[69]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[70]  Konstantinos N. Plataniotis,et al.  Restoration of Motion Blurred Images , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[71]  Stanley T. Birchfield,et al.  Elliptical head tracking using intensity gradients and color histograms , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[72]  C. Atkeson,et al.  Robot Catching , 2000 .

[73]  Oliver Birbach,et al.  A Multiple Hypothesis Approach for a Ball Tracking System , 2009, ICVS.

[74]  Anders Robertsson,et al.  Vision Based Tracker for Dart Catching Robot , 2009, SyRoCo.

[75]  Radu Horaud,et al.  Controlling robots with two cameras: how to do it properly , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[76]  Yoichi Sato,et al.  Recognition of a mechanical linkage based on occlusion-robust object tracking , 2003, Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, MFI2003..

[77]  Mika Laaksonen,et al.  Skin detection in video under changing illumination conditions , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[78]  Joris De Schutter,et al.  Constraint-based Task Specification and Estimation for Sensor-Based Robot Systems in the Presence of Geometric Uncertainty , 2007, Int. J. Robotics Res..

[79]  Stephen P. Boyd,et al.  CVXGEN: a code generator for embedded convex optimization , 2011, Optimization and Engineering.

[80]  Anders Robertsson,et al.  High-speed visual robot control using an optimal linearizing intensity-based filtering approach , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[81]  Anders Robertsson,et al.  Visual position tracking using dual quaternions with hand-eye motion constraints , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[82]  Collin Wang,et al.  Heuristics for assembly sequencing and relative magazine assignment for robotic assembly , 1998 .

[83]  Karl-Erik Årzén.  JGrafchart: Sequence Control and Procedure Handling in Java , 2002 .

[84]  Dorin Comaniciu,et al.  Mean shift analysis and applications , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[85]  D. Foster.  Color constancy , 2011, Vision Research.

[86]  Vincenzo Lippiello,et al.  Monocular Eye-In-Hand Robotic Ball Catching with Parabolic Motion Estimation , 2012, SyRoCo.

[87]  Jean-Jacques E. Slotine,et al.  Experiments in Robotic Catching , 1991, 1991 American Control Conference.

[88]  Neville Hogan,et al.  Impedance Control: An Approach to Manipulation , 1984, 1984 American Control Conference.

[89]  Alexander Dietrich,et al.  Catching flying balls with a mobile humanoid: System overview and design considerations , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[90]  Gregory D. Hager,et al.  Control methods for guidance virtual fixtures in compliant human-machine interfaces , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[91]  J. C. Hyland.  An iterated-extended Kalman filter algorithm for tracking surface and sub-surface targets , 2002, OCEANS '02 MTS/IEEE.

[92]  Fumio Miyazaki,et al.  Learning to the robot table tennis task-ball control & rally with a human , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[93]  Gerd Hirzinger,et al.  Flexible robot-assembly using a multi-sensory approach , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[94]  Oliver Birbach,et al.  Tracking of Ball Trajectories with a Free Moving Camera-Inertial Sensor , 2008, RoboCup.

[95]  Berthold Bäuml,et al.  Realtime perception for catching a flying ball with a mobile humanoid , 2011, 2011 IEEE International Conference on Robotics and Automation.

[96]  Tamio Arai,et al.  Increasing Efficiency of Force-Controlled Robotic Assembly , 2006 .

[97]  Ishwar K. Sethi,et al.  Feature Point Correspondence in the Presence of Occlusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[98]  Subhash Challa,et al.  Sensor Fusion based 3D Target Visual Tracking for Autonomous Vehicles with IMM , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[99]  Kiyoshi Ohishi,et al.  Hybrid control of force and position without force sensor , 1992, Proceedings of the 1992 International Conference on Industrial Electronics, Control, Instrumentation, and Automation.

[100]  Il Hong Suh,et al.  Disturbance observer based force control of robot manipulator without force sensor , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[101]  Per Hagander.  Operator Factorization and Other Aspects of the Analysis of Linear Systems , 1973 .

[102]  Mark H. Overmars,et al.  A Comparative Study of Probabilistic Roadmap Planners , 2002, WAFR.

[103]  Toshiyuki Murakami,et al.  Torque sensorless control in multidegree-of-freedom manipulator , 1993, IEEE Trans. Ind. Electron..