Rearrangement on Lattices with Pick-n-Swaps: Optimality Structures and Efficient Algorithms

We investigate a class of multi-object rearrangement problems in which a robotic manipulator, capable of carrying an item and making item swaps, is tasked to sort items stored in lattices in a time-optimal manner. We systematically analyze the intrinsic optimality structure, which is fairly rich and intriguing, under different levels of item distinguishability (fully labeled, where each item has a unique label, or partially labeled, where multiple items may be of the same type) and different lattice dimensions. Focusing on the most practical setting of one and two dimensions, we develop efficient (low polynomial time) algorithms that optimally perform rearrangements on 1D lattices under both fullyand partially-labeled settings. On the other hand, we prove that rearrangement on 2D and higher dimensional lattices becomes computationally intractable to optimally solve. Despite their NP-hardness, we are able to again develop efficient algorithms for 2D fullyand partially-labeled settings that are asymptotically optimal, in expectation, assuming that the initial configuration is randomly selected. Simulation studies confirm the effectiveness of our algorithms in comparison to greedy best-first approaches.

[1]  Kostas E. Bekris,et al.  Fast, High-Quality Dual-Arm Rearrangement in Synchronous, Monotone Tabletop Setups , 2018, WAFR.

[2]  R. Prim Shortest connection networks and some generalizations , 1957 .

[3]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[4]  Dieter Fox,et al.  PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes , 2017, Robotics: Science and Systems.

[5]  Alberto Rodriguez,et al.  Prehensile pushing: In-hand manipulation with push-primitives , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Zherong Pan,et al.  Decision Making in Joint Push-Grasp Action Space for Large-Scale Object Sorting , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Kenneth Y. Goldberg,et al.  Orienting polygonal parts without sensors , 1993, Algorithmica.

[8]  Mehmet Remzi Dogar,et al.  Planning with a Receding Horizon for Manipulation in Clutter Using a Learned Value Function , 2018, 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids).

[9]  Christos H. Papadimitriou,et al.  The Euclidean Traveling Salesman Problem is NP-Complete , 1977, Theor. Comput. Sci..

[10]  Max Warshauer,et al.  The locker puzzle , 2006 .

[11]  Donald B. Johnson,et al.  Priority Queues with Update and Finding Minimum Spanning Trees , 1975, Inf. Process. Lett..

[12]  Kevin M. Lynch,et al.  Dynamic Nonprehensile Manipulation: Controllability, Planning, and Experiments , 1999, Int. J. Robotics Res..

[13]  Emilio Frazzoli,et al.  Asymptotically Optimal Algorithms for One-to-One Pickup and Delivery Problems With Applications to Transportation Systems , 2012, IEEE Transactions on Automatic Control.

[14]  Xinyu Liu,et al.  Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics , 2017, Robotics: Science and Systems.

[15]  Joshua A. Haustein,et al.  Multi-Object Rearrangement with Monte Carlo Tree Search: A Case Study on Planar Nonprehensile Sorting , 2019, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[17]  Kostas E. Bekris,et al.  Complexity Results and Fast Methods for Optimal Tabletop Rearrangement with Overhand Grasps , 2017, Int. J. Robotics Res..

[18]  J. F. Pierce,et al.  ON THE TRUCK DISPATCHING PROBLEM , 1971 .

[19]  ChangHwan Kim,et al.  Efficient Obstacle Rearrangement for Object Manipulation Tasks in Cluttered Environments , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[20]  Kate Saenko,et al.  High precision grasp pose detection in dense clutter , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Danica Kragic,et al.  Data-Driven Grasp Synthesis—A Survey , 2013, IEEE Transactions on Robotics.

[22]  Wolfram Burgard,et al.  Learning to Singulate Objects using a Push Proposal Network , 2017, ISRR.

[23]  Siddhartha S. Srinivasa,et al.  A Framework for Push-Grasping in Clutter , 2011, Robotics: Science and Systems.

[24]  E. Rivlin,et al.  Practical pushing planning for rearrangement tasks , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[25]  Dinesh Manocha,et al.  Centralized path planning for multiple robots: Optimal decoupling into sequential plans , 2009, Robotics: Science and Systems.

[26]  Siddhartha S. Srinivasa,et al.  Kinodynamic randomized rearrangement planning via dynamic transitions between statically stable states , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[27]  E. J.,et al.  ON THE COMPLEXITY OF MOTION PLANNING FOR MULTIPLE INDEPENDENT OBJECTS ; PSPACE HARDNESS OF THE " WAREHOUSEMAN ' S PROBLEM " . * * ) , 2022 .

[28]  Alberto Rodriguez,et al.  Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[29]  Kostas E. Bekris,et al.  A self-supervised learning system for object detection using physics simulation and multi-view pose estimation , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[30]  Philippe Flajolet,et al.  Analytic Combinatorics , 2009 .

[31]  Anca D. Dragan,et al.  Robot grasping in clutter: Using a hierarchy of supervisors for learning from demonstrations , 2016, 2016 IEEE International Conference on Automation Science and Engineering (CASE).

[32]  Tamim Asfour,et al.  Manipulation Planning Among Movable Obstacles , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[33]  Kostas E. Bekris,et al.  Efficiently solving general rearrangement tasks: A fast extension primitive for an incremental sampling-based planner , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Changjoo Nam,et al.  Planning for target retrieval using a robotic manipulator in cluttered and occluded environments , 2019, ArXiv.

[35]  Kostas E. Bekris,et al.  Dealing with Difficult Instances of Object Rearrangement , 2015, Robotics: Science and Systems.

[36]  Siddhartha S. Srinivasa,et al.  Rearrangement planning using object-centric and robot-centric action spaces , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[37]  James J. Kuffner,et al.  Navigation among movable obstacles: real-time reasoning in complex environments , 2004, 4th IEEE/RAS International Conference on Humanoid Robots, 2004..

[38]  Ashutosh Saxena,et al.  Robotic Grasping of Novel Objects using Vision , 2008, Int. J. Robotics Res..

[39]  Leslie Pack Kaelbling,et al.  Hierarchical task and motion planning in the now , 2011, 2011 IEEE International Conference on Robotics and Automation.

[40]  Gert Kootstra,et al.  International Conference on Robotics and Automation (ICRA) , 2008, ICRA 2008.

[41]  Lydia E. Kavraki,et al.  Learning Feasibility for Task and Motion Planning in Tabletop Environments , 2019, IEEE Robotics and Automation Letters.

[42]  Gordon T. Wilfong Motion planning in the presence of movable obstacles , 1988, SCG '88.

[43]  Siddhartha S. Srinivasa,et al.  Extrinsic dexterity: In-hand manipulation with external forces , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[44]  Esra Erdem,et al.  Geometric rearrangement of multiple movable objects on cluttered surfaces: A hybrid reasoning approach , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[45]  Russell H. Taylor,et al.  Sensor-based manipulation planning as a game with nature , 1988 .

[46]  Abdeslam Boularias,et al.  Learning to Manipulate Unknown Objects in Clutter by Reinforcement , 2015, AAAI.

[47]  Siddhartha S. Srinivasa,et al.  Nonprehensile whole arm rearrangement planning on physics manifolds , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[48]  Mark Moll,et al.  Randomized Physics-Based Motion Planning for Grasping in Cluttered and Uncertain Environments , 2017, IEEE Robotics and Automation Letters.

[49]  Eric Huang,et al.  Large-Scale Multi-Object Rearrangement , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[50]  Kuan-Ting Yu,et al.  Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching , 2019, The International Journal of Robotics Research.

[51]  Richard M. Karp,et al.  Reducibility Among Combinatorial Problems , 1972, 50 Years of Integer Programming.

[52]  Baichuan Huang,et al.  DIPN: Deep Interaction Prediction Network with Application to Clutter Removal , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[53]  Dieter Fox,et al.  Interactive singulation of objects from a pile , 2012, 2012 IEEE International Conference on Robotics and Automation.

[54]  Dan Halperin,et al.  k-color multi-robot motion planning , 2012, Int. J. Robotics Res..

[55]  M. T. Mason,et al.  Toward Robotic Manipulation , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[56]  Anna Gál,et al.  The cell probe complexity of succinct data structures , 2007, Theor. Comput. Sci..