Real-time path planning for automating optical tweezers based particle transport operations

Title of Dissertation: REAL-TIME PATH PLANNING FOR AUTOMATING OPTICAL TWEEZERS BASED PARTICLE TRANSPORT OPERATIONS Ashis Gopal Banerjee, Doctor of Philosophy, 2009 Directed by: Professor Satyandra K. Gupta Department of Mechanical Engineering Optical tweezers (OT) have been developed to successfully trap, orient, and transport micro and nano scale components of many different sizes and shapes in a fluid medium. They can be viewed as robots made out of light. Components can be simply released from optical traps by switching off laser beams. By utilizing the principle of time sharing or holograms, multiple optical traps can perform several operations in parallel. These characteristics make optical tweezers a very promising technology for creating directed micro and nano scale assemblies. In the infra-red regime, they are useful in a large number of biological applications as well. This dissertation explores the problem of real-time path planning for autonomous OT based transport operations. Such operations pose interesting challenges as the environment is uncertain and dynamic due to the random Brownian motion of the particles and noise in the imaging based measurements. Silica microspheres having diameters between (1-20) μm are selected as model components. Offline simulations are performed to gather trapping probability data that serves as a measure of trap strength and reliability as a function of relative position of the particle under consideration with respect to the trap focus, and trap velocity. Simplified models are generated using Gaussian Radial Basis Functions to represent the data in a compact form. These metamodels can be queried at run-time to obtain estimated probability values accurately and efficiently. Simple trapping probability models are then utilized in a stochastic dynamic programming framework to compute optimum trap locations and velocities that minimizes the total, expected transport time by incorporating collision avoidance and recovery steps. A discrete version of an approximate partially observable Markov decision process algorithm, called the QMDP_NLTDV algorithm, is developed. Real-time performance is ensured by pruning the search space and enhancing convergence rates by introducing a non-linear value function. The algorithm is validated both using a simulator as well as a physical holographic tweezer set-up. Successful runs show that the automated planner is flexible, works well in reasonably crowded scenes, and is capable of transporting a specific particle to a given goal location by avoiding collisions either by circumventing or by trapping other freely diffusing particles. This technique for transporting individual particles is utilized within a decoupled and prioritized approach to move multiple particles simultaneously. An iterative version of a bipartite graph matching algorithm is also used to assign goal locations to target objects optimally. As in the case of single particle transport, simulation and some physical experiments are performed to validate the multi-particle planning approach. REAL-TIME PATH PLANNING FOR AUTOMATING OPTICAL TWEEZERS BASED PARTICLE TRANSPORT OPERATIONS

[1]  Steven M. LaValle,et al.  Algorithms for Planning under Uncertainty in Prediction and Sensing , 2006, Autonomous Mobile Robots.

[2]  Mark H. Overmars,et al.  Coordinated path planning for multiple robots , 1998, Robotics Auton. Syst..

[3]  H. Rubinsztein-Dunlop,et al.  Calculation of the T-matrix: general considerations and application of the point-matching method , 2003 .

[4]  Gregory S. Chirikjian,et al.  Steering flexible needles under Markov motion uncertainty , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007 .

[6]  Tomás Lozano-Pérez,et al.  Spatial Planning: A Configuration Space Approach , 1983, IEEE Transactions on Computers.

[7]  Mark de Berg,et al.  Computational Geometry: Algorithms and Applications, Second Edition , 2000 .

[8]  E. Stelzer,et al.  Optical trapping of dielectric particles in arbitrary fields. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[9]  M. Wang,et al.  Parametric Structural Shape and Topology Optimization Method With Radial Basis Functions and Level-Set Method , 2006, DAC 2006.

[10]  Gary Friedman,et al.  Magnetic separation, manipulation and assembly of solid phase in fluids , 2005 .

[11]  Aristides A. G. Requicha,et al.  Automated Nanomanipulation with Atomic Force Microscopes , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[12]  Lorenzo Sadun Applied Linear Algebra: The Decoupling Principle , 2000 .

[13]  Alexander Rohrbach,et al.  Trapping forces, force constants, and potential depths for dielectric spheres in the presence of spherical aberrations. , 2002, Applied optics.

[14]  S K Lee,et al.  Dielectrophoretic alignment of gallium nitride nanowires (GaN NWs) for use in device applications , 2006, Nanotechnology.

[15]  A. Ashkin Forces of a single-beam gradient laser trap on a dielectric sphere in the ray optics regime. , 1992, Methods in cell biology.

[16]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[17]  Richard K. Beatson,et al.  Reconstruction and representation of 3D objects with radial basis functions , 2001, SIGGRAPH.

[18]  Satyandra K. Gupta,et al.  A Flexible System Framework for a Nanoassembly Cell Using Optical Tweezers , 2006 .

[19]  Hiroo Ukita,et al.  Resolving Discrepancy between Theoretical and Experimental Optical Trapping Forces Using Effects of Beam Waist and Trapping Position Displacement due to Gravity , 2006 .

[20]  K Bergman,et al.  Characterization of photodamage to Escherichia coli in optical traps. , 1999, Biophysical journal.

[21]  Pramod N. Chivate,et al.  Review of surface representations and fitting for reverse engineering , 1995 .

[22]  Qi Xia,et al.  Implicit fitting and smoothing using radial basis functions with partition of unity , 2005, Ninth International Conference on Computer Aided Design and Computer Graphics (CAD-CG'05).

[23]  Nicholas Roy,et al.  Adapting probabilistic roadmaps to handle uncertain maps , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[24]  P. R. Ouyang,et al.  Micro-motion devices technology: The state of arts review , 2008 .

[25]  Douglas C. Montgomery,et al.  Applied Statistics and Probability for Engineers, Third edition , 1994 .

[26]  Woei Ming Lee,et al.  Optical Separation of Cells on Potential Energy Landscapes: Enhancement With Dielectric Tagging , 2007, IEEE Journal of Selected Topics in Quantum Electronics.

[27]  Mark H. Overmars,et al.  The Gaussian sampling strategy for probabilistic roadmap planners , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[28]  J. P. Barton,et al.  Internal and near‐surface electromagnetic fields for a spherical particle irradiated by a focused laser beam , 1988 .

[29]  Thierry Siméon,et al.  Path coordination for multiple mobile robots: a resolution-complete algorithm , 2002, IEEE Trans. Robotics Autom..

[30]  Ren-Jye Yang,et al.  Approximation methods in multidisciplinary analysis and optimization: a panel discussion , 2004 .

[31]  Steven M. Block,et al.  Optical trapping of metallic Rayleigh particles. , 1994, Optics letters.

[32]  Manuela M. Veloso,et al.  Real-Time Randomized Path Planning for Robot Navigation , 2002, RoboCup.

[33]  M W Berns,et al.  Parametric study of the forces on microspheres held by optical tweezers. , 1994, Applied optics.

[34]  Seth Hutchinson,et al.  A Framework for Real-time Path Planning in Changing Environments , 2002, Int. J. Robotics Res..

[35]  Isaac Amidror,et al.  Scattered data interpolation methods for electronic imaging systems: a survey , 2002, J. Electronic Imaging.

[36]  Qi Xia,et al.  3D Reconstruction Methods Based on Radial Basis Functions for Laser Scanned Data Point Sets , 2006 .

[37]  Mitchel Weissbluth,et al.  Photon-Atom Interactions , 1989 .

[38]  Ellips Masehian,et al.  Classic and Heuristic Approaches in Robot Motion Planning A Chronological Review , 2007 .

[39]  David S. Ebert,et al.  Hardware-assisted feature analysis and visualization of procedurally encoded multifield volumetric data , 2005, IEEE Computer Graphics and Applications.

[40]  Chi-Kuang Sun,et al.  Cell manipulation by use of diamond microparticles as handles of optical tweezers , 2001 .

[41]  Ting Yu,et al.  The manipulation and assembly of CuO nanorods with line optical tweezers , 2004 .

[42]  Martin A. M. Gijs,et al.  Magnetic bead handling on-chip: new opportunities for analytical applications , 2004 .

[43]  David Grier,et al.  Processing carbon nanotubes with holographic optical tweezers. , 2004, Optics express.

[44]  M. Sitti,et al.  Survey of nanomanipulation systems , 2001, Proceedings of the 2001 1st IEEE Conference on Nanotechnology. IEEE-NANO 2001 (Cat. No.01EX516).

[45]  D. Grier A revolution in optical manipulation , 2003, Nature.

[46]  Pal Ormos,et al.  Orientation of flat particles in optical tweezers by linearly polarized light. , 2003, Optics express.

[47]  James F. O'Brien,et al.  Modelling with implicit surfaces that interpolate , 2002, TOGS.

[48]  Srinivas Akella,et al.  Coordinating Multiple Robots with Kinodynamic Constraints Along Specified Paths , 2005, Int. J. Robotics Res..

[49]  Jean-Claude Latombe,et al.  Robot Motion Planning: A Distributed Representation Approach , 1991, Int. J. Robotics Res..

[50]  Nicholas Roy,et al.  The permutable POMDP: fast solutions to POMDPs for preference elicitation , 2008, AAMAS.

[51]  H. Rubinsztein-Dunlop,et al.  Optical alignment and spinning of laser-trapped microscopic particles , 1998, Nature.

[52]  Vladimir J. Lumelsky,et al.  Decentralized Motion Planning for Multiple Mobile Robots: The Cocktail Party Model , 1997, Auton. Robots.

[53]  Lydia E. Kavraki,et al.  Probabilistic roadmaps for path planning in high-dimensional configuration spaces , 1996, IEEE Trans. Robotics Autom..

[54]  A. Ardeshir Goshtasby,et al.  Grouping and parameterizing irregularly spaced points for curve fitting , 2000, TOGS.

[55]  Achille Messac,et al.  Extended Radial Basis Functions: More Flexible and Effective Metamodeling , 2004 .

[56]  Michael W. Berns,et al.  Radiation trapping forces on microspheres with optical tweezers , 1993 .

[57]  Steven M. LaValle,et al.  An Objective-Based Framework for Motion Planning under Sensing and Control Uncertainties , 1998, Int. J. Robotics Res..

[58]  A J Pons,et al.  Feedback control of unstable cellular solidification fronts. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[59]  Günther Schmidt,et al.  Conflict-free motion of multiple mobile robots based on decentralized motion planning and negotiation , 1997, Proceedings of International Conference on Robotics and Automation.

[60]  J. Banavar,et al.  Computer Simulation of Liquids , 1988 .

[61]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[62]  S. Chu,et al.  Observation of a single-beam gradient force optical trap for dielectric particles. , 1986, Optics letters.

[63]  Norman R. Heckenberg,et al.  Optical tweezers computational toolbox , 2007 .

[64]  Dinesh Manocha,et al.  Reactive deformation roadmaps: motion planning of multiple robots in dynamic environments , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[65]  R. Bashir,et al.  Dielectrophoretic separation and manipulation of live and heat-treated cells of Listeria on microfabricated devices with interdigitated electrodes , 2002 .

[66]  Dan Cojoc,et al.  Biological samples micro-manipulation by means of optical tweezers , 2005 .

[67]  Hejun Du,et al.  Fundamental principles and applications of microfluidic systems. , 2008, Frontiers in bioscience : a journal and virtual library.

[68]  Wendy C. Crone,et al.  Magnetic Manipulation of Copper−Tin Nanowires Capped with Nickel Ends , 2004 .

[69]  Kostas E. Bekris,et al.  Greedy but Safe Replanning under Kinodynamic Constraints , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[70]  Kishan Dholakia,et al.  Passive optical separation within a 'nondiffracting' light beam. , 2007, Journal of biomedical optics.

[71]  Dinesh Manocha,et al.  Interactive navigation of multiple agents in crowded environments , 2008, I3D '08.

[72]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[73]  Joe Cecil,et al.  Assembly and manipulation of micro devices-A state of the art survey , 2007 .

[74]  Ernst Meyer,et al.  Scanning Probe Microscopy: The Lab on a Tip , 2021 .

[75]  Mengsu Yang,et al.  Microfluidics technology for manipulation and analysis of biological cells , 2006 .

[76]  Howie Choset,et al.  Principles of Robot Motion: Theory, Algorithms, and Implementation ERRATA!!!! 1 , 2007 .

[77]  Jennifer E. Curtis,et al.  Dynamic holographic optical tweezers , 2002 .

[78]  Amitabh Varshney,et al.  Modelling and Rendering Large Volume Data with Gaussian Radial Basis Functions , 2007 .

[79]  M. Ozkan,et al.  Electro-optical platform for the manipulation of live cells , 2003 .

[80]  Dinesh Manocha,et al.  Real-time navigation of independent agents using adaptive roadmaps , 2007, VRST '07.

[81]  Nicholas Roy,et al.  The Belief Roadmap: Efficient Planning in Linear POMDPs by Factoring the Covariance , 2007, ISRR.

[82]  K. Svoboda,et al.  Biological applications of optical forces. , 1994, Annual review of biophysics and biomolecular structure.

[83]  Alexander Rohrbach,et al.  Stiffness of optical traps: quantitative agreement between experiment and electromagnetic theory. , 2005, Physical review letters.

[84]  James J. Kuffner,et al.  Multipartite RRTs for Rapid Replanning in Dynamic Environments , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[85]  M W Berns,et al.  Physiological monitoring of optically trapped cells: assessing the effects of confinement by 1064-nm laser tweezers using microfluorometry. , 1996, Biophysical journal.

[86]  Yan Wu,et al.  An Obstacle-Based Probabilistic Roadmap Method For Path Planning , 1996 .

[87]  Ashis Gopal Banerjee,et al.  Radial Basis Function Based Simplified Trapping Probability Models for Optical Tweezers , 2008 .

[88]  Jean-Claude Latombe,et al.  Stochastic Roadmap Simulation: An Efficient Representation and Algorithm for Analyzing Molecular Motion , 2003, J. Comput. Biol..

[89]  Steven M. LaValle,et al.  Robot Motion Planning: A Game-Theoretic Foundation , 2000, Algorithmica.

[90]  Adriana Fontes,et al.  Electromagnetic forces for an arbitrary optical trapping of a spherical dielectric. , 2006, Optics express.

[91]  J. P. Barton,et al.  Theoretical determination of net radiation force and torque for a spherical particle illuminated by a focused laser beam , 1989 .

[92]  Johannes Courtial,et al.  Assembly of 3-dimensional structures using programmable holographic optical tweezers. , 2004, Optics express.

[93]  Amos H. C. Ng,et al.  Information Fusion for Simulation Based Decision Support in manufacturing , 2005 .

[94]  Steven M. LaValle,et al.  Quasi-randomized path planning , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[95]  Jean-Claude Latombe,et al.  Robot motion planning , 1970, The Kluwer international series in engineering and computer science.

[96]  Jean-Claude Latombe,et al.  Randomized Kinodynamic Motion Planning with Moving Obstacles , 2002, Int. J. Robotics Res..

[97]  Nancy M. Amato,et al.  A framework for planning motion in environments with moving obstacles , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[98]  A. Ashkin,et al.  History of optical trapping and manipulation of small-neutral particle, atoms, and molecules , 2000, IEEE Journal of Selected Topics in Quantum Electronics.

[99]  Rajeev Sharma,et al.  On Motion Planning in Changing, Partially Predictable Environments , 1997, Int. J. Robotics Res..