CAD Model-based Tracking and 3D Visual-based Control for MEMS Microassembly

This paper investigates sequential robotic microassembly for the construction of 3D micro-electro-mechanical systems (MEMSs) structures using a 3D visual servoing approach. The previous solutions proposed in the literature for these kinds of problems are based on 2D visual control because of the lack of precise and robust 3D measures from the work scene. In this paper, the relevance of the real-time 3D visual tracking method and the 3D vision-based control law proposed is demonstrated. The 3D poses of the MEMSs are supplied in real-time by a computer-aided design model-based tracking algorithm. This algorithm is sufficiently accurate and robust to enable a precise regulation toward zero of the 3D error using the proposed pose-based visual servoing approach. Experiments on a microrobotic setup have been carried out to achieve assemblies of two or more 400 μm × 400 μm × 100 μm silicon micro-objects by their respective 97 μm × 97 μm × 100 μm notches with an assembly clearance from 1 μm to 5 μm. The different microassembly processes are performed with a mean error of 0.3 μm in position and 0.35×10 −2 rad in orientation.

[1]  Sounkalo Dembélé,et al.  A multiscale calibration of a photon video microscope for visual servo control: Application to micromanipulation , 2008, 2008 International Workshop on Robotic and Sensors Environments.

[2]  Bradley J. Nelson,et al.  A CAD model based tracking system for visually guided microassembly , 2005, Robotica.

[3]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[4]  Rajagopalan Devanathan,et al.  Multi View and Multi Scale Image Based Visual Servo For Micromanipulation , 2005, Innovations in Robot Mobility and Control.

[5]  Bradley J. Nelson,et al.  Real-time rigid-body visual tracking in a scanning electron microscope , 2007 .

[6]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[7]  Daniel Jasper High-speed position tracking for nanohandling inside scanning electron microscopes , 2009, 2009 IEEE International Conference on Robotics and Automation.

[8]  Sergej Fatikow,et al.  Automatic control system of a microrobot-based microassembly station using computer vision , 1999, Optics East.

[9]  Sun Lining,et al.  Hybrid Control of Vision and Force for MEMS Assembly System , 2004 .

[10]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[11]  B. Nelson,et al.  Calibration of a parametric model of an optical microscope , 1999 .

[12]  Wolfgang Birkfellner,et al.  A fully automated calibration method for an optical see-through head-mounted operating microscope with variable zoom and focus , 2005, IEEE Transactions on Medical Imaging.

[13]  Yu Zhou,et al.  Force and Vision Feedback for Robotic Manipulation of the Microworld , 1999, ISER.

[14]  William J. Wilson,et al.  Relative end-effector control using Cartesian position based visual servoing , 1996, IEEE Trans. Robotics Autom..

[15]  Quan Zhou,et al.  Microhandling using Robotic Manipulation and Capillary Self-alignment , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Roberto Cipolla,et al.  Real-Time Visual Tracking of Complex Structures , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Xinhua Zhuang,et al.  Pose estimation from corresponding point data , 1989, IEEE Trans. Syst. Man Cybern..

[18]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[19]  John T. Feddema,et al.  Visual servoing and CAD-driven microassembly , 1998, IEEE Robotics Autom. Mag..

[20]  Bradley J. Nelson,et al.  Optomechatronic design of microassembly systems for manufacturing hybrid microsystems , 2005, IEEE Transactions on Industrial Electronics.

[21]  Kenneth A. Goldberg,et al.  Microassembly technologies for MEMS , 1998, Photonics West - Micro and Nano Fabricated Electromechanical and Optical Components.

[22]  Éric Marchand,et al.  Real-time markerless tracking for augmented reality: the virtual visual servoing framework , 2006, IEEE Transactions on Visualization and Computer Graphics.

[23]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[24]  Bradley J. Nelson,et al.  Micropositioning of a weakly calibrated microassembly system using coarse-to-fine visual servoing strategies , 2000 .

[25]  Michel Perrier,et al.  Controlling an Uninstrumented Manipulator By Visual Servoing , 2002, Int. J. Robotics Res..

[26]  W.T. Sun,et al.  Image-based visual servo for micromanipulation: a multiple-view and multiple-scale approach , 2004, Micro-Nanomechatronics and Human Science, 2004 and The Fourth Symposium Micro-Nanomechatronics for Information-Based Society, 2004..

[27]  Heikki N. Koivo,et al.  Environmental influences on microassembly , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  C. Dahmen,et al.  Towards automated robot-based nanohandling , 2009, 2009 Chinese Control and Decision Conference.

[29]  F. Arai,et al.  3D attitude control system for bio-micromanipulation , 2001, MHS2001. Proceedings of 2001 International Symposium on Micromechatronics and Human Science (Cat. No.01TH8583).

[30]  Seth Hutchinson,et al.  Visual Servo Control Part I: Basic Approaches , 2006 .

[31]  Bradley J. Nelson,et al.  A Supervisory Wafer-Level 3D Microassembly System for Hybrid MEMS Fabrication , 2003, J. Intell. Robotic Syst..

[32]  Sounkalo Dembélé,et al.  Automatic micromanipulation using multiscale visual servoing , 2008, 2008 IEEE International Conference on Automation Science and Engineering.

[33]  Jake J. Abbott,et al.  Robotics in the Small, Part I: Microbotics , 2007, IEEE Robotics & Automation Magazine.

[34]  Bradley J. Nelson,et al.  A Deformable Object Tracking Algorithm Robust to Occlusions and Spurious Edges , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[35]  Mehdi Ammi,et al.  Flexible Microscope Calibration using Virtual Pattern for 3-D Telemicromanipulation , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[36]  Hyungsuck Cho,et al.  Microassembly of peg and hole using active zooming , 2005, International Symposium on Optomechatronic Technologies.

[37]  Brahim Tamadazte,et al.  A Multiscale Calibration of a Photon Videomicroscope for Visual Servo Control: Application to MEMS Micromanipulation and Microassembly , 2009 .

[38]  Christoph Hürzeler,et al.  A Microassembly System for the Flexible Assembly of Hybrid Robotic Mems Devices , 2009 .

[39]  Mehdi Ammi,et al.  Robotic Assisted Micromanipulation System using Virtual Fixtures and Metaphors , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[40]  David G. Lowe,et al.  Fitting Parameterized Three-Dimensional Models to Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  E. Malis,et al.  2 1/2 D Visual Servoing , 1999 .

[42]  Quan Zhou,et al.  Influence of environment conditions on microassembly components and pick-and-place operations , 2002 .

[43]  David S. Ebert,et al.  Visualization and computer graphics , 2007 .

[44]  Nicolas Chaillet,et al.  Overview of Microgrippers and Design of a Micromanipulation Station Based on a MMOC Microgripper , 2005, 2005 International Symposium on Computational Intelligence in Robotics and Automation.

[45]  S. Hutchinson,et al.  Visual Servo Control Part II : Advanced Approaches , 2007 .

[46]  Bradley J. Nelson,et al.  Robust CAD model based visual tracking for 3D microassembly using image space potentials , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[47]  Philippe Martinet,et al.  Position based visual servoing using a non-linear approach , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[48]  Dan O. Popa,et al.  Micro and Mesoscale Robotic Assembly , 2004 .

[49]  François Chaumette,et al.  Visual servo control. II. Advanced approaches [Tutorial] , 2007, IEEE Robotics & Automation Magazine.

[50]  S. A. Stoeter,et al.  Improving Tracking Precision for Microassembly , 2006 .

[51]  Bradley J. Nelson,et al.  Vision-based force measurement , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.