Cloud-Based Networked Visual Servo Control

The performance of vision-based control systems, in particular, of highly dynamic vision-based motion control systems, is often limited by the low sampling rate of the visual feedback caused by the long image processing time. In order to overcome this problem, the networked visual servo control (NVSC), which integrates networked computational resources for cloud image processing, is considered in this paper. The main contributions of this paper are the following: 1) a real-time transport protocol for transmitting large-volume image data on a cloud-computing platform, which enables high-sampling-rate visual feedback; 2) a stabilizing control law for the NVSC system with time-varying feedback time delay; and 3) a sending rate scheduling strategy aiming at reducing the communication network load. The performance of the NVSC system with sending rate scheduling is validated in an object-tracking scenario on a 14-DOF dual-arm robot. Experimental results show the superior performance of our approach. In particular, the communication network load is substantially reduced by means of the scheduling strategy without performance degradation.

[1]  Nicholas R. Gans,et al.  Stable Visual Servoing Through Hybrid Switched-System Control , 2007, IEEE Transactions on Robotics.

[2]  Vern Paxson,et al.  TCP Congestion Control , 1999, RFC.

[3]  Sandra Hirche,et al.  Performance-oriented networked visual servo control with sending rate scheduling , 2011, 2011 IEEE International Conference on Robotics and Automation.

[4]  Mark S. Nixon,et al.  Feature Extraction and Image Processing , 2002 .

[5]  Yoshiaki Shirai,et al.  Guiding a robot by visual feedback in assembling tasks , 1973, Pattern Recognit..

[6]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[7]  Sandra Hirche,et al.  A framework of networked visual servo control system with distributed computation , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[8]  Mo-Yuen Chow,et al.  Networked Control System: Overview and Research Trends , 2010, IEEE Transactions on Industrial Electronics.

[9]  A. Hassibi,et al.  Control with random communication delays via a discrete-time jump system approach , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[10]  H. Buttner,et al.  Real-time ethernet the EtherCAT solution , 2004 .

[11]  S. Hirche,et al.  Stability, stabilization and experiments for networked control systems with random time delay , 2008, 2008 American Control Conference.

[12]  Sandra Hirche,et al.  Distributed computation and data scheduling for networked visual servo control systems , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Kang G. Shin,et al.  Computing time delay and its effects on real-time control systems , 1995, IEEE Trans. Control. Syst. Technol..

[14]  Koichi Hashimoto,et al.  Visual Servoing: Real-Time Control of Robot Manipulators Based on Visual Sensory Feedback , 1993 .

[15]  Sandra Hirche,et al.  Guaranteed Cost Control over Quality-of-Service Networks , 2009 .

[16]  Rahul Sukthankar,et al.  Distributed localization of networked cameras , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[17]  Lee E. Weiss,et al.  Dynamic sensor-based control of robots with visual feedback , 1987, IEEE Journal on Robotics and Automation.

[18]  Yskandar Hamam,et al.  Optimal integrated control and scheduling of networked control systems with communication constraints: application to a car suspension system , 2006, IEEE Transactions on Control Systems Technology.

[19]  橋本 浩一.  Visual servoing : real-time control of robot manipulators based on visual sensory feedback , 1993 .

[20]  Rajkumar Buyya,et al.  High Performance Cluster Computing: Architectures and Systems , 1999 .

[21]  Henning Schulzrinne,et al.  RTP Profile for Audio and Video Conferences with Minimal Control , 2003, RFC.

[22]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[23]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[24]  Sandra Hirche,et al.  On LQG joint optimal scheduling and control under communication constraints , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[25]  Hong Ye,et al.  Scheduling of networked control systems , 2001 .

[26]  Sandra Hirche,et al.  Structural characterization of optimal event-based controllers for linear stochastic systems , 2010, 49th IEEE Conference on Decision and Control (CDC).

[27]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[28]  Eduardo Tovar,et al.  Real-time fieldbus communications using Profibus networks , 1999, IEEE Trans. Ind. Electron..

[29]  S. Hirche,et al.  Performance Oriented Control over Networks -- Switching Controllers and Switched Time Delay -- , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[30]  K. Åström,et al.  Comparison of Riemann and Lebesgue sampling for first order stochastic systems , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[31]  Emilia Fridman,et al.  Robust sampled-data stabilization of linear systems: an input delay approach , 2004, Autom..

[32]  Masatoshi Ishikawa,et al.  High speed grasping using visual and force feedback , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[33]  Sandra Hirche,et al.  Controller design and experimental validation for networked control systems with time-varying random delay (特集 ネットワーク化制御システムの展開) , 2008 .

[34]  E. Malis,et al.  2 1/2 D Visual Servoing , 1999 .

[35]  Gianluca Cena,et al.  An improved CAN fieldbus for industrial applications , 1997, IEEE Trans. Ind. Electron..

[36]  François Chaumette,et al.  Image moments: a general and useful set of features for visual servoing , 2004, IEEE Transactions on Robotics.

[37]  R. Court.  Real-time Ethernet , 1992, Comput. Commun..

[38]  Sandra Hirche,et al.  Switching Control for a Networked Vision-based Control System , 2011, Autom..

[39]  D. Henrich,et al.  Parallel processing approaches in robotics , 1997, ISIE '97 Proceeding of the IEEE International Symposium on Industrial Electronics.

[40]  Tai C Yang,et al.  Networked control system: a brief survey , 2006 .

[41]  Sandra Hirche,et al.  On the Optimality of Certainty Equivalence for Event-Triggered Control Systems , 2013, IEEE Transactions on Automatic Control.

[42]  Tingting Xu,et al.  A high-speed multi-GPU implementation of bottom-up attention using CUDA , 2009, 2009 IEEE International Conference on Robotics and Automation.

[43]  Bartlomiej Stanczyk.  Development and Control of an Anthropomorphic Telerobotic System , 2006 .

[44]  Sandra Hirche,et al.  Towards Quality-of-Service control of networked control systems: A switched time delay systems approach , 2006, 2006 IEEE Conference on Computer Aided Control System Design, 2006 IEEE International Conference on Control Applications, 2006 IEEE International Symposium on Intelligent Control.

[45]  James Moyne,et al.  Using deadbands to reduce communication in networked control systems , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[46]  Peter Corke,et al.  VISUAL CONTROL OF ROBOT MANIPULATORS – A REVIEW , 1993 .

[47]  W. Richard Stevens,et al.  TCP/IP Illustrated, Volume 2: The Implementation , 1995 .

[48]  Bruno Siciliano,et al.  Modelling and Control of Robot Manipulators , 1997, Advanced Textbooks in Control and Signal Processing.

[49]  Vaidy S. Sunderam,et al.  PVM: A Framework for Parallel Distributed Computing , 1990, Concurr. Pract. Exp..

[50]  A. Cervin,et al.  Optimal On-line Sampling Period Assignment for Real-Time Control Tasks Based on Plant State Information , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[51]  Martin Buss,et al.  An explorative study of visual servo control with insect-inspired Reichardt-model , 2009, 2009 IEEE International Conference on Robotics and Automation.

[52]  Sandra Hirche,et al.  A switching control law for a networked visual servo control system , 2010, 2010 IEEE International Conference on Robotics and Automation.

[53]  Binoy Ravindran,et al.  Time-utility function-driven switched Ethernet: packet scheduling algorithm, implementation, and feasibility analysis , 2004, IEEE Transactions on Parallel and Distributed Systems.

[54]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[55]  Sandra Hirche,et al.  TOWARDS DEADBAND CONTROL IN NETWORKED TELEOPERATION SYSTEMS , 2005 .

[56]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[57]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[58]  Tongwen Chen,et al.  A new method for stabilization of networked control systems with random delays , 2005 .

[59]  Bruce A. Francis,et al.  Optimal Sampled-Data Control Systems , 1996, Communications and Control Engineering Series.

[60]  Sandra Hirche,et al.  Sampled-data Networked Control Systems with Random Time Delay , 2008 .

[61]  Peter I. Corke,et al.  Dynamic effects in visual closed-loop systems , 1996, IEEE Trans. Robotics Autom..

[62]  Rs Roel Pieters,et al.  Visual Servo Control , 2012 .

[63]  Martin Buss,et al.  An FPGA implementation of insect-inspired motion detector for high-speed vision systems , 2008, 2008 IEEE International Conference on Robotics and Automation.

[64]  Kolja Kühnlenz,et al.  Visual servoing using triangulation with an omnidirectional multi-camera system , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[65]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[66]  J. Feld,et al.  PROFINET - scalable factory communication for all applications , 2004, IEEE International Workshop on Factory Communication Systems, 2004. Proceedings..

[67]  Chi-kin Lee.  Distributed image processing on a network of workstations , 1994 .

[68]  Panos J. Antsaklis,et al.  Stability of model-based networked control systems with time-varying transmission times , 2004, IEEE Transactions on Automatic Control.