Control of autonomous robot teams in industrial applications

The use of teams of coordinated mobile robots in industrial settings such as underground mining, toxic waste cleanup and material storage and handling, is a viable and reliable approach to solving such problems that require or involve automation. In this thesis, abilities a team of mobile robots should demonstrate in order to successfully perform a mission in industrial settings are identified as a set of functional components. These components are related to navigation and obstacle avoidance, localization, task achieving behaviors and mission planning. The thesis focuses on designing and developing functional components applicable to diverse missions involving teams of mobile robots; in detail, the following are presented: (1) A navigation and obstacle avoidance technique to safely navigate the robot in an unknown environment. The technique relies on information retrieved by the robot’s vision system and sonar sensors to identify and avoid surrounding obstacles. (2) A localization method based on Kalman filtering and Fuzzy logic to estimate the robot’s position. The method uses information derived by multiple robot sensors such as vision system, odometer, laser range finder, GPS and IMU. (3) A target tracking and collision avoidance technique based on information derived by a vision system and a laser range finder. The technique is applicable in scenarios where an intruder is identified in the patrolling area. (4) A limited lookahead control methodology responsible for mission planning. The methodology is based on supervisory control theory and it is responsible for task allocation between the robots of the team. The control methodology considers situations where a robot may fail during operation. The performance of each functional component has been verified through extensive experimentation in indoor and outdoor environments. As a case study, a warehouse patrolling application is considered to demonstrate the effectiveness of the mission planning component.

[1]  George A. Bekey,et al.  AUTONOMOUS ROBOTS, From Biological Inspiration to Implementation and Control, by G.A. Bekey, MIT Press, 2005, xv + 577 pp., index, ISBN 0-262-02578-7, 25 pages of references (Hb. £35.95) , 2005, Robotica.

[2]  Rachid Alami,et al.  M+: a scheme for multi-robot cooperation through negotiated task allocation and achievement , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[3]  Toshiyuki Murakami,et al.  An approach to tracking motion of mobile robot for moving object , 2000, 2000 26th Annual Conference of the IEEE Industrial Electronics Society. IECON 2000. 2000 IEEE International Conference on Industrial Electronics, Control and Instrumentation. 21st Century Technologies.

[4]  Gaurav S. Sukhatme,et al.  Robust localization using relative and absolute position estimates , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[5]  M. Isabel Ribeiro,et al.  Active range sensing for mobile robot localization , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[6]  Anna Helena Reali Costa,et al.  A hybrid adaptive architecture for mobile robots based on reactive behaviors , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[7]  F. Ade,et al.  Using the condensation algorithm to implement tracking for mobile robots , 1999, 1999 Third European Workshop on Advanced Mobile Robots (Eurobot'99). Proceedings (Cat. No.99EX355).

[8]  Horst-Michael Groß,et al.  Contribution to vision-based localization, tracking and navigation methods for an interactive mobile service-robot , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[9]  Avinash C. Kak,et al.  Vision-based navigation by a mobile robot with obstacle avoidance using single-camera vision and ultrasonic sensing , 1998, IEEE Trans. Robotics Autom..

[10]  Kiichiro Izumida,et al.  A combination of monocular CCD camera and inertial-sensor for range estimation , 2002, IEEE 2002 28th Annual Conference of the Industrial Electronics Society. IECON 02.

[11]  Subhasis Chaudhuri,et al.  Depth estimation using defocused stereo image pairs , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[12]  Kimon P. Valavanis,et al.  Mobile robot navigation in 2-D dynamic environments using an electrostatic potential field , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[13]  B. Krishnamurthy,et al.  HelpMate: A robotic courier for hospital use , 1992, [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics.

[14]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[15]  Hichem Sahli,et al.  SENSOR INTEGRATION ON A MOBILE ROBOT , 2002 .

[16]  James J. Little,et al.  Spinoza: A stereoscopic visually guided mobile robot , 1999 .

[17]  Ratnesh Kumar,et al.  Extension based Limited Lookahead Supervision of Discrete Event Systems , 1998, Autom..

[18]  Kiriakos Kiriakidis,et al.  Supervision of multiple-robot systems , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[19]  P. J. Escamilla-Ambrosio,et al.  A hybrid Kalman filter-fuzzy logic architecture for multisensor data fusion , 2001, Proceeding of the 2001 IEEE International Symposium on Intelligent Control (ISIC '01) (Cat. No.01CH37206).

[20]  Penny Probert Smith,et al.  LICAs: a modular architecture for intelligent control of mobile robots , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[21]  A. Tiano,et al.  Application of interval and fuzzy techniques to integrated navigation systems , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[22]  G. Whelan,et al.  Cooperative search and rescue with a team of mobile robots , 1997, 1997 8th International Conference on Advanced Robotics. Proceedings. ICAR'97.

[23]  John C. Russ,et al.  The Image Processing Handbook , 2016, Microscopy and Microanalysis.

[24]  Michail G. Lagoudakis,et al.  Simple auctions with performance guarantees for multi-robot task allocation , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[25]  Yan Yongjie,et al.  Hybrid Control Architecture of Mobile Robot Based on Subsumption Architecture , 2006, 2006 International Conference on Mechatronics and Automation.

[26]  Ray G. Gosine,et al.  Coordinated execution of tasks in a multiagent environment , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[27]  Parvaneh Saeedi,et al.  3D motion tracking of a mobile robot in a natural environment , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[28]  James S. Albus,et al.  NASA/NBS Standard Reference Model for Telerobot Control System Architecture (NASREM) , 1989 .

[29]  S. Chung,et al.  Limited lookahead policies in supervisory control of discrete event systems , 1992 .

[30]  Erol Sahin,et al.  Mobile robot range sensing through visual looming , 1998, Proceedings of the 1998 IEEE International Symposium on Intelligent Control (ISIC) held jointly with IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA) Intell.

[31]  Tzuu-Hseng S. Li,et al.  An experimental study on tracking control of two autonomous mobile robots , 1997, Proceedings of the IECON'97 23rd International Conference on Industrial Electronics, Control, and Instrumentation (Cat. No.97CH36066).

[32]  Kimon P. Valavanis,et al.  Mobile Robot Navigation Using Sonar and Range Measurements from Uncalibrated Cameras , 2006, 2006 14th Mediterranean Conference on Control and Automation.

[33]  Shigeru Tsuchiya,et al.  A fault-tolerant control algorithm having a decentralized autonomous architecture for space hyper-redundant manipulators , 1998, IEEE Trans. Syst. Man Cybern. Part A.

[34]  Norihiko Adachi,et al.  Image-based visual adaptive tracking control of nonholonomic mobile robots , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[35]  Jing Wang,et al.  DRS operating primitives based on distributed mutual exclusion , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[36]  Aldo Cipriano,et al.  Fuzzy logic based nonlinear Kalman filter applied to mobile robots modelling , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[37]  E. Gat On Three-Layer Architectures , 1997 .

[38]  A. V. Timofeev,et al.  Hybrid algorithms of multi-agent control of mobile robots , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[39]  Viii Supervisor Sonar-Based Real-World Mapping and Navigation , 2001 .

[40]  Lynne E. Parker,et al.  ALLIANCE: an architecture for fault tolerant multirobot cooperation , 1998, IEEE Trans. Robotics Autom..

[41]  Fernando Matia A Novel Fuzzy Kalman Filter for Mobile Robots Localization , 2004 .

[42]  Paolo Remagnino,et al.  Multi-camera colour tracking , 1999, Proceedings Second IEEE Workshop on Visual Surveillance (VS'99) (Cat. No.98-89223).

[43]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Nejib Ben Hadj-Alouane,et al.  Variable lookahead supervisory control with state information , 1994 .

[45]  Wolfram Burgard,et al.  Principles of Robot Motion: Theory, Algorithms, and Implementation ERRATA!!!! 1 , 2007 .

[46]  Hani Hagras,et al.  A hierarchical type-2 fuzzy logic control architecture for autonomous mobile robots , 2004, IEEE Transactions on Fuzzy Systems.

[47]  Diana F. Spears,et al.  Reconfigurable robot teams: modeling and supervisory control , 2004, IEEE Transactions on Control Systems Technology.

[48]  Kunikatsu Takase,et al.  Robust tracking for camera control on an irregular terrain vehicle , 2002, Proceedings of the 41st SICE Annual Conference. SICE 2002..

[49]  Jian Shen,et al.  Target-tracking and path planning for vehicle following in jungle environment , 2004, ICARCV 2004 8th Control, Automation, Robotics and Vision Conference, 2004..

[50]  John N. Tsitsiklis,et al.  A survey of computational complexity results in systems and control , 2000, Autom..

[51]  A. Kandel,et al.  Multiple sensor based UGV localization using fuzzy extended Kalman filtering , 2007, 2007 Mediterranean Conference on Control & Automation.

[52]  Michael J. Swain,et al.  Indexing via color histograms , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[53]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[54]  S. Lafortune,et al.  Recursive computation of limited lookahead supervisory controls for discrete event systems , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[55]  David Fernández Llorca,et al.  AUTOPIA architecture for automatic driving and maneuvering , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[56]  Feng Lin,et al.  How to reuse supervisors when discrete event system models evolve , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[57]  Feng Lin,et al.  On-line control of partially observed discrete event systems , 1994, Discret. Event Dyn. Syst..

[58]  Ioannis Pitas,et al.  Segmentation and tracking of faces in color images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[59]  L. Farkas,et al.  An intelligent integrated sensor system for the ORNL mobile robot , 1988, Proceedings IEEE International Symposium on Intelligent Control 1988.

[60]  Karen Rudie,et al.  Near-Optimal Online Control of Dynamic Discrete-Event Systems , 2006, Discret. Event Dyn. Syst..

[61]  Randall Davis,et al.  Negotiation as a Metaphor for Distributed Problem Solving , 1988, Artificial Intelligence.

[62]  Robin R. Murphy,et al.  Application of the Distributed Field Robot Architecture to a Simulated Demining Task , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[63]  Kimon P. Valavanis,et al.  Fuzzy logic based autonomous skid steering vehicle navigation , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[64]  M. Kleinehagenbrock,et al.  Person tracking with a mobile robot based on multi-modal anchoring , 2002, Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication.

[65]  Gregory M. P. O'Hare,et al.  Sensor fusion for social robotics , 2000 .

[66]  Pascal Vasseur,et al.  A Vision Algorithm for Dynamic Detection of Moving Vehicles with a UAV , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[67]  Subhasis Chaudhuri,et al.  Depth estimation and image restoration using defocused stereo pairs , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[68]  Manuela M. Veloso,et al.  A Modular Hierarchical Behavior-Based Architecture , 2001, RoboCup.

[69]  Stéphane Lafortune,et al.  Supervisory control using variable lookahead policies , 1993, 1993 American Control Conference.

[70]  Ching-Chih Tsai A localization system of a mobile robot by fusing dead-reckoning and ultrasonic measurements , 1998, IEEE Trans. Instrum. Meas..

[71]  H. Chou,et al.  A simultaneous localization and mapping algorithm based on Kalman filtering , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[72]  Bruce A. Draper,et al.  Color machine vision for autonomous vehicles , 1998 .

[73]  Bum-Jae You,et al.  A real-time color-based object tracking robust to irregular illumination variations , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[74]  Roland Siegwart,et al.  Multisensor on-the-fly localization using laser and vision , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[75]  Raúl Enrique Sánchez-Yáñez,et al.  Fuzzy color tracking for robotic tasks , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[76]  Fag Lin,et al.  Robust and Adaptive Supervisory Control of Discrete Event Systems , 1992, 1992 American Control Conference.

[77]  Tzuu-Hseng S. Li,et al.  Realization of two-dimensional target tracking problem via autonomous mobile robots using fuzzy sliding mode control , 1998, IECON '98. Proceedings of the 24th Annual Conference of the IEEE Industrial Electronics Society (Cat. No.98CH36200).

[78]  Rajnikant V. Patel,et al.  On-line robot trajectory planning for catching a moving object , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[79]  Tzuu-Hseng S. Li,et al.  Fuzzy target tracking control of autonomous mobile robots by using infrared sensors , 2004, IEEE Transactions on Fuzzy Systems.

[80]  Hugh F. Durrant-Whyte,et al.  Inertial navigation systems for mobile robots , 1995, IEEE Trans. Robotics Autom..

[81]  Vijay K. Garg,et al.  Control of stochastic discrete event systems modeled by probabilistic languages , 2001, IEEE Trans. Autom. Control..

[82]  W. Sardha Wijesoma,et al.  A laser and a camera for mobile robot navigation , 2002, 7th International Conference on Control, Automation, Robotics and Vision, 2002. ICARCV 2002..

[83]  Patrick Rives,et al.  Robustness of image-based visual servoing with respect to depth distribution errors , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[84]  Kai-Tai Song,et al.  Environment perception for a mobile robot using double ultrasonic sensors and a CCD camera , 1996, IEEE Trans. Ind. Electron..

[85]  Myung Hwangbo,et al.  A stable target-tracking control for unicycle mobile robots , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[86]  Sheng-Jyh Wang,et al.  Contrast based color segmentation with adaptive thresholds , 2002, Proceedings. International Conference on Image Processing.

[87]  Yann Chevaleyre,et al.  Theoretical analysis of the multi-agent patrolling problem , 2004, Proceedings. IEEE/WIC/ACM International Conference on Intelligent Agent Technology, 2004. (IAT 2004)..

[88]  Peter Stone,et al.  Individual and collaborative behaviors in a team of homogeneous robotic soccer agents , 1998, Proceedings International Conference on Multi Agent Systems (Cat. No.98EX160).

[89]  Kimon P. Valavanis,et al.  Vision Based Target Tracking and Collision Avoidance for Mobile Robots , 2007, J. Intell. Robotic Syst..

[90]  Anders Orebäck,et al.  BERRA: a research architecture for service robots , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[91]  Fan Jiang,et al.  Depth from dynamic stereo images , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[92]  Paul S. Schenker,et al.  CAMPOUT: a control architecture for tightly coupled coordination of multirobot systems for planetary surface exploration , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[93]  K. Kiriakidis,et al.  Adaptive supervisory control of interconnected discrete event systems , 2000, Proceedings of the 2000. IEEE International Conference on Control Applications. Conference Proceedings (Cat. No.00CH37162).

[94]  Shang-Hong Lai,et al.  A Generalized Depth Estimation Algorithm with a Single Image , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[95]  Maja J. Mataric,et al.  Sold!: auction methods for multirobot coordination , 2002, IEEE Trans. Robotics Autom..

[96]  Asok Ray,et al.  A behavior-based collaborative multi-agent system , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[97]  Ferat Sahin,et al.  Cognitive maps in swarm robots for the mine detection application , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[98]  W. D. Rencken Autonomous Sonar Navigation in Indoor, Unknown and Unstructured Environments , 1994, IROS.

[99]  Hubert Roth,et al.  Mobile robots for search and rescue , 2005, IEEE International Safety, Security and Rescue Rototics, Workshop, 2005..

[100]  Hajime Asama,et al.  Design Of An Autonomous And Distributed Robot System: Actress , 1989, Proceedings. IEEE/RSJ International Workshop on Intelligent Robots and Systems '. (IROS '89) 'The Autonomous Mobile Robots and Its Applications.

[101]  S. Niwa,et al.  Kalman filter with time-variable gain for a multisensor fusion system , 1999, Proceedings. 1999 IEEE/SICE/RSJ. International Conference on Multisensor Fusion and Integration for Intelligent Systems. MFI'99 (Cat. No.99TH8480).

[102]  Eric T. Baumgartner,et al.  An autonomous vision-based mobile robot , 1994, IEEE Trans. Autom. Control..

[103]  Zhu Changan,et al.  Obstacle detection using adaptive color segmentation and planar projection stereopsis for mobile robots , 2003, IEEE International Conference on Robotics, Intelligent Systems and Signal Processing, 2003. Proceedings. 2003.

[104]  Johan Forsberg,et al.  Mobile robot localization: integrating measurements from a time-of-flight laser , 1996, IEEE Trans. Ind. Electron..

[105]  P. Ramadge,et al.  Supervisory control of a class of discrete event processes , 1987 .

[106]  Ali Yalcin,et al.  Dynamic Task Allocation in Cooperative Robot Teams , 2009 .

[107]  Manuel Mazo,et al.  Fusing odometric and vision data with an EKF to estimate the absolute position of an autonomous mobile robot , 2003, EFTA 2003. 2003 IEEE Conference on Emerging Technologies and Factory Automation. Proceedings (Cat. No.03TH8696).

[108]  Ying Wu,et al.  Nonstationary color tracking for vision-based human-computer interaction , 2002, IEEE Trans. Neural Networks.

[109]  Christian Balkenius,et al.  Visual tracking and target selection for mobile robots , 1996, Proceedings of the First Euromicro Workshop on Advanced Mobile Robots (EUROBOT '96).

[110]  P. Hartana,et al.  Sensor data fusion using Kalman filter , 2000, Proceedings of the Third International Conference on Information Fusion.

[111]  Brian V. Funt,et al.  Color Constant Color Indexing , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[112]  Ren C. Luo,et al.  Autonomous mobile target tracking system based on grey-fuzzy control algorithm , 2000, IEEE Trans. Ind. Electron..

[113]  Mark Yim,et al.  Indoor automation with many mobile robots , 1990, EEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications.

[114]  Kyoung-Ho Choi,et al.  An advanced approach for navigation and image sensor integration for land vehicle navigation , 2004, IEEE 60th Vehicular Technology Conference, 2004. VTC2004-Fall. 2004.

[115]  Howie Choset,et al.  Probabilistic methods for robotic landmine search , 2001, SPIE Optics East.

[116]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[117]  Don Ray Murray,et al.  Cooperative robot localization with vision-based mapping , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[118]  Narciso García,et al.  Face detection based on a new color space YCgCr , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[119]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[120]  Trevor Darrell,et al.  Integrated Person Tracking Using Stereo, Color, and Pattern Detection , 2000, International Journal of Computer Vision.

[121]  Anthony Stentz,et al.  Multi-robot exploration controlled by a market economy , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[122]  James L. Crowley,et al.  Position estimation for a mobile robot using vision and odometry , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[123]  Roland Stenzel A behavior-based control architecture , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[124]  Dongbing Gu,et al.  Landmark-based navigation of mobile robots in manufacturing , 1999, 1999 7th IEEE International Conference on Emerging Technologies and Factory Automation. Proceedings ETFA '99 (Cat. No.99TH8467).

[125]  E. H. Mamdani,et al.  Advances in the linguistic synthesis of fuzzy controllers , 1976 .

[126]  Ettore Stella,et al.  Position estimation for a mobile robot using data fusion , 1995, Proceedings of Tenth International Symposium on Intelligent Control.

[127]  Christos G. Cassandras,et al.  Introduction to Discrete Event Systems, Second Edition , 2008 .

[128]  Kazunori Umeda,et al.  Subpixel stereo method: a new methodology of stereo vision , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).