Visual Servoing of Nonholonomic Mobile Robots With Uncalibrated Camera-to-Robot Parameters

In this paper, a monocular visual servoing strategy is presented for nonholonomic mobile robots. Different from existing methods, the proposed approach works well even with both unknown extrinsic camera-to-robot parameters and unknown depth. Considering these unknown parameters, the mathematical model of an eye-in-hand mobile robot becomes an uncertain nonholonomic system which violates the conventional triangularity condition, as a result, the stabilization problem is very challenging and still unsolved. Therefore, a novel two-stage controller is proposed by using adaptive control and backstepping methods. In the first stage, adaptive velocity controllers are carefully designed to drive the angular error and the lateral error to an arbitrarily small neighborhood of zero, and asymptotic convergence is proven using Lyapunov-based techniques. Afterwards, a proportional controller is applied to regulate the longitudinal error in the second stage. Experimental results are provided to verify the effectiveness of the proposed approach.