Control strategies for cleaning robots in domestic applications: A comprehensive review

Service robots are built and developed for various applications to support humans as companion, caretaker, or domestic support. As the number of elderly people grows, service robots will be in increasing demand. Particularly, one of the main tasks performed by elderly people, and others, is the complex task of cleaning. Therefore, cleaning tasks, such as sweeping floors, washing dishes, and wiping windows, have been developed for the domestic environment using service robots or robot manipulators with several control approaches. This article is primarily focused on control methodology used for cleaning tasks. Specifically, this work mainly discusses classical control and learning-based controlled methods. The classical control approaches, which consist of position control, force control, and impedance control , are commonly used for cleaning purposes in a highly controlled environment. However, classical control methods cannot be generalized for cluttered environment so that learning-based control methods could be an alternative solution. Learning-based control methods for cleaning tasks can encompass three approaches: learning from demonstration (LfD), supervised learning (SL), and reinforcement learning (RL). These control approaches have their own capabilities to generalize the cleaning tasks in the new environment. For example, LfD, which many research groups have used for cleaning tasks, can generate complex cleaning trajectories based on human demonstration. Also, SL can support the prediction of dirt areas and cleaning motion using large number of data set. Finally, RL can learn cleaning actions and interact with the new environment by the robot itself. In this context, this article aims to provide a general overview of robotic cleaning tasks based on different types of control methods using manipulator. It also suggest a description of the future directions of cleaning tasks based on the evaluation of the control approaches.

[1]  Nikolaos G. Tsagarakis,et al.  OpenSoT: A whole-body control library for the compliant humanoid robot COMAN , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[2]  L. P. Kaelbling,et al.  Learning Symbolic Models of Stochastic Domains , 2007, J. Artif. Intell. Res..

[3]  Xi Chen,et al.  Bilateral teleoperation system for a musculoskeletal robot arm using a musculoskeletal exoskeleton , 2017, 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[4]  Krishnanand N. Kaipa,et al.  Robotic bimanual cleaning of deformable objects with online learning of part and tool models , 2016, 2016 IEEE International Conference on Automation Science and Engineering (CASE).

[5]  Carme Torras,et al.  Planning robot manipulation to clean planar surfaces , 2015, Eng. Appl. Artif. Intell..

[6]  Cecilia Laschi,et al.  Soft robotics: a bioinspired evolution in robotics. , 2013, Trends in biotechnology.

[7]  Darwin G. Caldwell,et al.  On improving the extrapolation capability of task-parameterized movement models , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Masayuki Inaba,et al.  Multi-cue 3D object recognition in knowledge-based vision-guided humanoid robot system , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Jun Morimoto,et al.  Adaptation and coaching of periodic motion primitives through physical and visual interaction , 2016, Robotics Auton. Syst..

[10]  Carme Torras,et al.  Incremental Learning of Skills in a Task-Parameterized Gaussian Mixture Model , 2015, Journal of Intelligent & Robotic Systems.

[11]  Charles C. Kemp,et al.  Challenges for robot manipulation in human environments [Grand Challenges of Robotics] , 2007, IEEE Robotics & Automation Magazine.

[12]  Tamim Asfour,et al.  Learn to wipe: A case study of structural bootstrapping from sensorimotor experience , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Carlos Andrey Maia,et al.  A new algebraic approach for the description of robotic manipulation tasks , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Francis L. Merat,et al.  Introduction to robotics: Mechanics and control , 1987, IEEE J. Robotics Autom..

[15]  Sachin Chitta,et al.  MoveIt! [ROS Topics] , 2012, IEEE Robotics Autom. Mag..

[16]  Daniel Leidner,et al.  Robotic Agents Representing, Reasoning, and Executing Wiping Tasks for Daily Household Chores , 2016, AAMAS.

[17]  Darwin G. Caldwell,et al.  Reinforcement Learning in Robotics: Applications and Real-World Challenges , 2013, Robotics.

[18]  Peng Hao,et al.  Transfer learning using computational intelligence: A survey , 2015, Knowl. Based Syst..

[19]  Andrej Gams,et al.  On-line coaching of robots through visual and physical interaction: Analysis of effectiveness of human-robot interaction strategies , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Wolfram Burgard,et al.  Null space optimization for effective coverage of 3D surfaces using redundant manipulators , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Matteo Cianchetti,et al.  Soft robotics: Technologies and systems pushing the boundaries of robot abilities , 2016, Science Robotics.

[22]  Filip Ilievski,et al.  Soft robotics for chemists. , 2011, Angewandte Chemie.

[23]  Maya Cakmak,et al.  Towards a comprehensive chore list for domestic robots , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[24]  Keiji Nagatani,et al.  Motion planning for dual-arm mobile manipulator - realization of "tidying a room motion" , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[25]  Dragomir N. Nenchev,et al.  Experimental evaluation of a trajectory/force tracking controller for a humanoid robot cleaning a vertical surface , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Miomir Vukobratovic,et al.  Zero-Moment Point - Thirty Five Years of its Life , 2004, Int. J. Humanoid Robotics.

[27]  Ales Ude,et al.  Action sequencing using dynamic movement primitives , 2011, Robotica.

[28]  Fumiya Iida,et al.  The challenges ahead for bio-inspired 'soft' robotics , 2012, CACM.

[29]  S. Schaal Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics , 2006 .

[30]  Gregory D. Hager,et al.  An incremental approach to learning generalizable robot tasks from human demonstration , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Carme Torras,et al.  Planning Clearing Actions in Cluttered Scenes by Phasing in Geometrical Constraints , 2017, CCIA.

[32]  Haruhisa Kawasaki,et al.  Dexterous anthropomorphic robot hand with distributed tactile sensor: Gifu hand II , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[33]  Stefan Leutenegger,et al.  Deep learning a grasp function for grasping under gripper pose uncertainty , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[34]  Aude Billard,et al.  Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models , 2011, IEEE Transactions on Robotics.

[35]  Andrej Gams,et al.  Learning and adaptation of periodic motion primitives based on force feedback and human coaching interaction , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[36]  Daniel Leidner,et al.  Inferring the effects of wiping motions based on haptic perception , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[37]  Masayuki Inaba,et al.  Task guided attention control and visual verification in tea serving by the daily assistive humanoid HRP2JSK , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  Tamim Asfour,et al.  Learning and force adaptation for interactive actions , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[39]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..

[40]  Christian Dornhege,et al.  Integrated Symbolic Planning in the Tidyup-Robot Project , 2013, AAAI Spring Symposium: Designing Intelligent Robots.

[41]  Ali Sadeghi,et al.  Modular Continuum Manipulator: Analysis and Characterization of Its Basic Module , 2018, Biomimetics.

[42]  R.V. Patel,et al.  A Robust Position and Force Control Strategy for 7-DOF Redundant Manipulators , 2009, IEEE/ASME Transactions on Mechatronics.

[43]  Ron Alterovitz,et al.  Demonstration-Guided Motion Planning , 2011, ISRR.

[44]  Jun Li,et al.  Dual quaternion based kinematic control for Yumi dual arm robot , 2017, 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[45]  Peter K. Allen,et al.  Graspit! A versatile simulator for robotic grasping , 2004, IEEE Robotics & Automation Magazine.

[46]  Stefan Wermter,et al.  Training Agents With Interactive Reinforcement Learning and Contextual Affordances , 2016, IEEE Transactions on Cognitive and Developmental Systems.

[47]  Alin Albu-Schäffer,et al.  Learning from demonstration: repetitive movements for autonomous service robotics , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[48]  Rouhollah Rahmatizadeh,et al.  Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-to-End Learning from Demonstration , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[49]  You Zhou,et al.  Vision-Based Online Adaptation of Motion Primitives to Dynamic Surfaces: Application to an Interactive Robotic Wiping Task , 2018, IEEE Robotics and Automation Letters.

[50]  Ales Ude,et al.  Enhancing the performance of adaptive iterative learning control with reinforcement learning , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[51]  W. Burgard,et al.  Learning the State Transition Model to Efficiently Clean Surfaces with Mobile Manipulation Robots , 2011 .

[52]  MajidiCarmel,et al.  Soft Robotics: A Perspective—Current Trends and Prospects for the Future , 2014 .

[53]  Andrej Gams,et al.  On-line periodic movement and force-profile learning for adaptation to new surfaces , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[54]  Affan Pervez,et al.  Learning task-parameterized dynamic movement primitives using mixture of GMMs , 2018, Intell. Serv. Robotics.

[55]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[56]  Darwin G. Caldwell,et al.  Learning bimanual end-effector poses from demonstrations using task-parameterized dynamical systems , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[57]  Tomoaki Nakamura,et al.  Visual Recognition System for Cleaning Tasks by Humanoid Robots , 2013 .

[58]  Bernhard Nebel,et al.  How Much Does a Household Robot Need to Know in Order to Tidy Up , 2013, AAAI 2013.

[59]  Krishnanand N. Kaipa,et al.  Automated Planning for Robotic Cleaning Using Multiple Setups and Oscillatory Tool Motions , 2017, IEEE Transactions on Automation Science and Engineering.

[60]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[61]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[62]  Sergey Levine,et al.  Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[63]  Daniel Leidner,et al.  Classifying compliant manipulation tasks for automated planning in robotics , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[64]  Peter K. Allen,et al.  Task level hierarchical system for BCI-enabled shared autonomy , 2017, 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).

[65]  Joni Pajarinen,et al.  Robotic manipulation of multiple objects as a POMDP , 2014, Artif. Intell..

[66]  Sungchul Kang,et al.  A novel method for estimating external force: Simulation study with a 4-DOF robot manipulator , 2015 .

[67]  Maren Bennewitz,et al.  Mobile manipulation in cluttered environments with humanoids: Integrated perception, task planning, and action execution , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[68]  Hal Hodson Robot cleaner can empty bins and sweep floors , 2015 .

[69]  Martin Hägele,et al.  New brooms sweep clean - an autonomous robotic cleaning assistant for professional office cleaning , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[70]  Carme Torras,et al.  Relational reinforcement learning with guided demonstrations , 2017, Artif. Intell..

[71]  Wolfram Burgard,et al.  Imitation learning with generalized task descriptions , 2009, 2009 IEEE International Conference on Robotics and Automation.

[72]  Chih-Hung King,et al.  Towards an assistive robot that autonomously performs bed baths for patient hygiene , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[73]  Paolo Fiorini,et al.  A Short History of Cleaning Robots , 2000, Auton. Robots.

[74]  Gennaro Raiola,et al.  Iterative virtual guides programming for human-robot comanipulation , 2017, 2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[75]  Tamim Asfour,et al.  Coordinate Change Dynamic Movement Primitives — A leader-follower approach , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[76]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[77]  Chad DeChant,et al.  Shape completion enabled robotic grasping , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[78]  Masayuki Shimizu,et al.  Analytical inverse kinematics for 5-DOF humanoid manipulator under arbitrarily specified unconstrained orientation of end-effector , 2014, Robotica.

[79]  John J. Craig,et al.  Hybrid position/force control of manipulators , 1981 .

[80]  Sergey Levine,et al.  Deep Object-Centric Representations for Generalizable Robot Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[81]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[82]  Ali Sadeghi,et al.  SIMBA: Tendon-Driven Modular Continuum Arm with Soft Reconfigurable Gripper , 2017, Front. Robot. AI.

[83]  Dongheui Lee,et al.  Incremental kinesthetic teaching of motion primitives using the motion refinement tube , 2011, Auton. Robots.

[84]  Gordon Cheng,et al.  Extracting general task structures to accelerate the learning of new tasks , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[85]  Neville Hogan,et al.  Impedance Control: An Approach to Manipulation , 1984, 1984 American Control Conference.

[86]  Marc Toussaint,et al.  Planning with Noisy Probabilistic Relational Rules , 2010, J. Artif. Intell. Res..

[87]  Kimitoshi Yamazaki,et al.  System integration of a daily assistive robot and its application to tidying and cleaning rooms , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[88]  Affan Pervez,et al.  Learning deep movement primitives using convolutional neural networks , 2017, 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).

[89]  Alexandre Bernardino,et al.  Autonomous table-cleaning from kinesthetic demonstrations using Deep Learning , 2018, 2018 Joint IEEE 8th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[90]  Yoshihiko Nakamura,et al.  Inverse kinematic solutions with singularity robustness for robot manipulator control , 1986 .

[91]  Ian D. Walker,et al.  Soft robotics: Biological inspiration, state of the art, and future research , 2008 .

[92]  José Santos-Victor,et al.  “iCub, clean the table!” A robot learning from demonstration approach using deep neural networks , 2018, 2018 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC).

[93]  Darwin G. Caldwell,et al.  Upper-body kinesthetic teaching of a free-standing humanoid robot , 2011, 2011 IEEE International Conference on Robotics and Automation.

[94]  Gisbert Lawitzky A Navigation System for Cleaning Robots , 2000, Auton. Robots.

[95]  Valerio Ortenzi,et al.  An experimental study of robot control during environmental contacts based on projected operational space dynamics , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[96]  Christopher G. Atkeson,et al.  Physical human interaction for an inflatable manipulator , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[97]  Darwin G. Caldwell,et al.  Learning and Reproduction of Gestures by Imitation , 2010, IEEE Robotics & Automation Magazine.

[98]  Alain Micaelli,et al.  Dynamic balance control of humanoids for multiple grasps and non coplanar frictional contacts , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[99]  Darwin G. Caldwell,et al.  Learning from demonstrations with partially observable task parameters , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[100]  Krishnanand N. Kaipa,et al.  Automated Learning of Operation Parameters for Robotic Cleaning by Mechanical Scrubbing , 2016 .

[101]  Sungchul Kang,et al.  External force estimation using joint torque sensors and its application to impedance control of a robot manipulator , 2013, 2013 13th International Conference on Control, Automation and Systems (ICCAS 2013).

[102]  Sonia Chernova,et al.  Towards Robot Adaptability in New Situations , 2015, AAAI Fall Symposia.

[103]  Aude Billard,et al.  BM: An iterative algorithm to learn stable non-linear dynamical systems with Gaussian mixture models , 2010, 2010 IEEE International Conference on Robotics and Automation.

[104]  Carme Torras,et al.  Safe robot execution in model-based reinforcement learning , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[105]  Alois Knoll,et al.  Compliant control for soft robots: Emergent behavior of a tendon driven anthropomorphic arm , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[106]  Masayuki Inaba,et al.  Vision based behavior verification system of humanoid robot for daily environment tasks , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[107]  Michael Beetz,et al.  Multidimensional Time-Series Shapelets Reliably Detect and Classify Contact Events in Force Measurements of Wiping Actions , 2018, IEEE Robotics and Automation Letters.

[108]  Yasemin Altun,et al.  Relative Entropy Policy Search , 2010 .

[109]  Maya Cakmak,et al.  Learning generalizable surface cleaning actions from demonstration , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[110]  Fumiya Iida,et al.  Soft Manipulators and Grippers: A Review , 2016, Front. Robot. AI.

[111]  D.J. Reinkensmeyer,et al.  A pneumatic robot for re-training arm movement after stroke: rationale and mechanical design , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[112]  Ken Perlin,et al.  [Computer Graphics]: Three-Dimensional Graphics and Realism , 2022 .

[113]  Stefan Schaal,et al.  Encoding of periodic and their transient motions by a single dynamic movement primitive , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[114]  Andrew W. Moore,et al.  Locally Weighted Learning for Control , 1997, Artificial Intelligence Review.

[115]  Daniel Leidner,et al.  Knowledge-enabled parameterization of whole-body control strategies for compliant service robots , 2016, Auton. Robots.

[116]  Stefano Stramigioli,et al.  The Safety of Domestic Robotics: A Survey of Various Safety-Related Publications , 2014, IEEE Robotics & Automation Magazine.