Asynchronous Event-Based Visual Shape Tracking for Stable Haptic Feedback in Microrobotics

Micromanipulation systems have recently been receiving increased attention. Teleoperated or automated micromanipulation is a challenging task due to the need for high-frequency position or force feedback to guarantee stability. In addition, the integration of sensors within micromanipulation platforms is complex. Vision is a commonly used solution for sensing; unfortunately, the update rate of the frame-based acquisition process of current available cameras cannot ensure-at reasonable costs-stable automated or teleoperated control at the microscale level, where low inertia produces highly unreachable dynamic phenomena. This paper presents a novel vision-based microrobotic system combining both an asynchronous address event representation silicon retina and a conventional frame-based camera. Unlike frame-based cameras, recent artificial retinas transmit their outputs as a continuous stream of asynchronous temporal events in a manner similar to the output cells of a biological retina, enabling high update rates. This paper introduces an event-based iterative closest point algorithm to track a microgripper's position at a frequency of 4 kHz. The temporal precision of the asynchronous silicon retina is used to provide a haptic feedback to assist users during manipulation tasks, whereas the frame-based camera is used to retrieve the position of the object that must be manipulated. This paper presents the results of an experiment on teleoperating a sphere of diameter around 50 μm using a piezoelectric gripper in a pick-and-place task.

[1]  T. Delbruck,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < 1 , 2022 .

[2]  Michaël Gauthier,et al.  Principle of a Submerged Freeze Gripper for Microassembly , 2008, IEEE Transactions on Robotics.

[3]  Misha Anne Mahowald,et al.  VLSI analogs of neuronal visual processing: a synthesis of form and function , 1992 .

[4]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[5]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  G. K. Ananthasuresh,et al.  Miniature Compliant Grippers With Vision-Based Force Sensing , 2010, IEEE Transactions on Robotics.

[7]  Xinyu Liu,et al.  Micronewton force-controlled manipulation of biomaterials using a monolithic MEMS microgripper with two-axis force feedback , 2008, 2008 IEEE International Conference on Robotics and Automation.

[8]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[9]  Y L Zhang,et al.  Automatic control of mechanical forces acting on cell biomembranes using a vision‐guided microrobotic system in computer microscopy , 2009, Journal of microscopy.

[10]  Tobias Delbrück,et al.  Frame-free dynamic digital vision , 2008 .

[11]  Sergej Fatikow,et al.  Force sensing in microrobotic systems-an overview , 1998 .

[12]  Stéphane Régnier,et al.  Microrobotics for Micromanipulation. , 2010 .

[13]  Tomaso Poggio,et al.  Models of object recognition , 2000, Nature Neuroscience.

[14]  B. Nelson,et al.  A Six-Axis MEMS Force–Torque Sensor With Micro-Newton and Nano-Newtonmeter Resolution , 2009, Journal of Microelectromechanical Systems.

[15]  Quan Zhou,et al.  Hybrid Microassembly Combining Robotics and Water Droplet Self-Alignment , 2010, IEEE Transactions on Robotics.

[16]  Ryad Benosman,et al.  Asynchronous event‐based high speed vision for microparticle tracking , 2012 .

[17]  Michaël Gauthier,et al.  Silicon end-effectors for microgripping tasks , 2009 .

[18]  Michael A. Greenspan,et al.  A high speed iterative closest point tracker on an FPGA platform , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[19]  J. F. Creemer,et al.  Electrothermal microgripper with large jaw displacement and integrated force sensors , 2008, 2008 IEEE 21st International Conference on Micro Electro Mechanical Systems.

[20]  Yu Sun,et al.  Autonomous Robotic Pick-and-Place of Microobjects , 2010, IEEE Transactions on Robotics.

[21]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[22]  Eugenio Culurciello,et al.  Activity-driven, event-based vision sensors , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[23]  J. Edward Colgate,et al.  Passivity of a class of sampled-data systems: Application to haptic interfaces , 1997, J. Field Robotics.

[24]  Bradley J. Nelson,et al.  Vision-based force measurement , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Aude Bolopion,et al.  Haptic Teleoperation for 3-D Microassembly of Spherical Objects , 2012, IEEE/ASME Transactions on Mechatronics.

[26]  Aude Bolopion,et al.  Variable gain haptic coupling for molecular simulation , 2011, 2011 IEEE World Haptics Conference.

[27]  Javier González,et al.  Mobile robot motion estimation by 2D scan matching with genetic and iterative closest point algorithms , 2006, J. Field Robotics.

[28]  C. P. A C O R E T,et al.  Asynchronous event-based high speed vision for microparticle tracking , 2011 .

[29]  V. Chawda,et al.  Vision-based force sensing for nanomanipulation , 2011, IEEE/ASME Transactions on Mechatronics.

[30]  K. Mølhave,et al.  Multimodal Electrothermal Silicon Microgrippers for Nanotube Manipulation , 2009, IEEE Transactions on Nanotechnology.

[31]  Vijay Kumar,et al.  Two-dimensional, vision-based μN force sensor for microrobotics , 2009, 2009 IEEE International Conference on Robotics and Automation.