A Review of Haptic Feedback Teleoperation Systems for Micromanipulation and Microassembly

This paper presents a review of the major haptic feedback teleoperation systems for micromanipulation. During the last decade, the handling of micrometer-sized objects has become a critical issue. Fields of application from material science to electronics demonstrate an urgent need for intuitive and flexible manipulation systems able to deal with small-scale industrial projects and assembly tasks. Two main approaches have been considered: fully automated tasks and manual operation. The first one require fully pre determined tasks, while the later necessitates highly trained operators. To overcome these issues the use of haptic feedback teleoperation where the user manipulates the tool through a joystick whilst feeling a force feedback, appears to be a promising solution as it allows high intuitiveness and flexibility. Major advances have been achieved during this last decade, starting with systems that enable the operator to feel the substrate topology, to the current state-of-the-art where 3D haptic feedback is provided to aid manipulation tasks. This paper details the major achievements and the solutions that have been developed to propose 3D haptic feedback for tools that often lack 3D force measurements. The use of virtual reality to enhance the immersion is also addressed. The strategies developed provide haptic feedback teleoperation systems with a high degree of assistance and for a wide range of micromanipulation tools. Based on this expertise on haptic for micromanipulation and virtual reality assistance it is now possible to propose microassembly systems for objects as small as 1 to 10 micrometers. This is a mature field and will benefit small-scale industrial projects where precision and flexibility in microassembly are required.

[1]  F. B. Llewellyn,et al.  Some Fundamental Properties of Transmission Systems , 1952, Proceedings of the IRE.

[2]  Y. Hatamura,et al.  Direct coupling system between nanometer world and human world , 1990, IEEE Proceedings on Micro Electro Mechanical Systems, An Investigation of Micro Structures, Sensors, Actuators, Machines and Robots..

[3]  S. Salcudean,et al.  Toward a tele-nanorobotic manipulation system with atomic scale force feedback and motion resolution , 1990, IEEE Proceedings on Micro Electro Mechanical Systems, An Investigation of Micro Structures, Sensors, Actuators, Machines and Robots..

[4]  J. Edward Colgate,et al.  Robust impedance shaping telemanipulation , 1993, IEEE Trans. Robotics Autom..

[5]  Tsuneo Yoshikawa,et al.  Analysis of maneuverability and stability of micro-teleoperation systems , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[6]  Septimiu E. Salcudean,et al.  Towards a Force-Reflecting Motion-Scaling System for Microsurgery , 1994, ICRA.

[7]  Quan Zhou,et al.  Fuzzy control system for microtelemanipulation , 1996 .

[8]  Russell M. Taylor,et al.  Pearls found on the way to the ideal interface for scanned probe microscopes , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[9]  Falvo,et al.  Nanomanipulation Experiments Exploring Frictional and Mechanical Properties of Carbon Nanotubes , 1998, Microscopy and Microanalysis.

[10]  Sergej Fatikow,et al.  Force sensing in microrobotic systems-an overview , 1998 .

[11]  Yu Zhou,et al.  Fusing force and vision feedback for micromanipulation , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[12]  Hideki Hashimoto,et al.  Macro to nano tele-manipulation through nanoelectromechanical systems , 1998, IECON '98. Proceedings of the 24th Annual Conference of the IEEE Industrial Electronics Society (Cat. No.98CH36200).

[13]  Hideki Hashimoto,et al.  Tele-nanorobotics using atomic force microscope , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[14]  Russell M. Taylor,et al.  Controlled manipulation of molecular samples with the nanoManipulator , 1999, 1999 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (Cat. No.99TH8399).

[15]  W. G. Matthews,et al.  Controlled manipulation of molecular samples with the nanoManipulator , 2000 .

[16]  Charles Baur,et al.  Delta Haptic Device as a nanomanipulator , 2001, Optics East.

[17]  Bradley J. Nelson,et al.  Sensing nanonewton level forces by visually tracking structural deformations , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[18]  Metin Sitti,et al.  Teleoperated touch feedback from the surfaces at the nanoscale: modeling and experiments , 2003 .

[19]  D. S. Haliyo,et al.  [mü]MAD, the adhesion based dynamic micro-manipulator , 2003 .

[20]  Annie Luciani,et al.  PRESENCE: the sense of believability of inaccessible worlds , 2003, Proceedings. 2003 International Conference on Cyberworlds.

[21]  Christos E. Constantinou,et al.  Intuitive micromanipulation with haptic audio feedback , 2004, The Fourth International Conference onComputer and Information Technology, 2004. CIT '04..

[22]  Bradley J. Nelson,et al.  Vision-based force measurement , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Gentiane Venture,et al.  Force-feedback micromanipulation with unconditionally stable coupling , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Yunhui Liu,et al.  Modeling of nanomanipulation with an integrated teleoperated system , 2005, 2005 IEEE International Conference on Robotics and Biomimetics - ROBIO.

[25]  Ning Xi,et al.  Modeling and Control of Active End Effector for the AFM Based Nano Robotic Manipulators , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[26]  Annie Luciani,et al.  Educational tool for nanophysics using multisensory rendering , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[27]  M. Sitti,et al.  Augmented reality user interface for an atomic force microscope-based nanorobotic system , 2006, IEEE Transactions on Nanotechnology.

[28]  Metin Sitti,et al.  Task-based and stable telenanomanipulation in a nanoscale virtual environment , 2006, IEEE Transactions on Automation Science and Engineering.

[29]  Antoine Ferreira,et al.  Virtual reality and haptics for nanorobotics , 2006, IEEE Robotics & Automation Magazine.

[30]  N. Xi,et al.  Development of A Haptic User Interface for Surface Sensing and Nanomanipulation Based on Atomic Force Microscope , 2006, 2006 1st IEEE International Conference on Nano/Micro Engineered and Molecular Systems.

[31]  Moussa Boukhnifer,et al.  Wave-based passive control for transparent micro-teleoperation system , 2006, Robotics Auton. Syst..

[32]  Cagdas D. Onal,et al.  Visual Servoing-Based Autonomous 2-D Manipulation of Microparticles Using a Nanoprobe , 2007, IEEE Transactions on Control Systems Technology.

[33]  Mehdi Ammi,et al.  Robotic Assisted Micromanipulation System using Virtual Fixtures and Metaphors , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[34]  Moussa Boukhnifer,et al.  $H_{\infty }$ Loop Shaping Bilateral Controller for a Two-Fingered Tele-Micromanipulation System , 2007, IEEE Transactions on Control Systems Technology.

[35]  Xinyu Liu,et al.  Micronewton force-controlled manipulation of biomaterials using a monolithic MEMS microgripper with two-axis force feedback , 2008, 2008 IEEE International Conference on Robotics and Automation.

[36]  Anatole Lécuyer,et al.  Improving Perception and Understanding of Nanoscale Phenomena Using Haptics and Visual Analogy , 2008, EuroHaptics.

[37]  Michaël Gauthier,et al.  Principle of a Submerged Freeze Gripper for Microassembly , 2008, IEEE Transactions on Robotics.

[38]  Aude Bolopion,et al.  Analysis of stability and transparency for nanoscale force feedback in bilateral coupling , 2008 .

[39]  J. F. Creemer,et al.  Electrothermal microgripper with large jaw displacement and integrated force sensors , 2008, 2008 IEEE 21st International Conference on Micro Electro Mechanical Systems.

[40]  D. S. Haliyo,et al.  Parallel imaging/manipulation force microscopy , 2009 .

[41]  K. Mølhave,et al.  Multimodal Electrothermal Silicon Microgrippers for Nanotube Manipulation , 2009, IEEE Transactions on Nanotechnology.

[42]  Aude Bolopion,et al.  2D micro teleoperation with force feedback , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[43]  B. Nelson,et al.  A Six-Axis MEMS Force–Torque Sensor With Micro-Newton and Nano-Newtonmeter Resolution , 2009, Journal of Microelectromechanical Systems.

[44]  Cagdas D. Onal,et al.  A Scaled Bilateral Control System for Experimental One-dimensional Teleoperated Nanomanipulation , 2009, Int. J. Robotics Res..

[45]  Stéphane Régnier,et al.  Microrobotics for Micromanipulation. , 2010 .

[46]  Lu Ren,et al.  Automated 3-D Micrograsping Tasks Performed by Vision-Based Control , 2010, IEEE Transactions on Automation Science and Engineering.

[47]  Stephane Regnier,et al.  Robotic Microassembly: Gauthier/Robotic Micro-Assembly , 2010 .

[48]  G. K. Ananthasuresh,et al.  Miniature Compliant Grippers With Vision-Based Force Sensing , 2010, IEEE Transactions on Robotics.

[49]  M. Sitti,et al.  Teleoperated 3-D Force Feedback From the Nanoscale With an Atomic Force Microscope , 2010, IEEE Transactions on Nanotechnology.

[50]  Philippe Lutz,et al.  Towards micro-assembly of hybrid MOEMS components on a reconfigurable silicon free-space micro-optical bench , 2010 .

[51]  Vijay Kumar,et al.  A two dimensional vision-based force sensor for microrobotic applications , 2011 .

[52]  Hui Xie,et al.  Development of a Flexible Robotic System for Multiscale Applications of Micro/Nanoscale Manipulation and Assembly , 2011, IEEE/ASME Transactions on Mechatronics.

[53]  Aude Bolopion,et al.  Vision-Based Haptic Feedback for Remote Micromanipulation in–SEM Environment , 2012 .

[54]  Ryad Benosman,et al.  Asynchronous Event-Based Visual Shape Tracking for Stable Haptic Feedback in Microrobotics , 2012, IEEE Transactions on Robotics.

[55]  Aude Bolopion,et al.  Haptic Teleoperation for 3-D Microassembly of Spherical Objects , 2012, IEEE/ASME Transactions on Mechatronics.

[56]  Adha Imam Cahyadi,et al.  Robotic Micro-Assembly , 2012 .