Reinforcement learning for landmark-based robot navigation

In landmark-based navigation, robots start in an unknown locationand must navigate to a desired target using visually-acquired land-marks. In the scenario that we are studying the target isvisible fromthe robot’s initial location, but it may subsequently be occludedby intervening objects. The challenge for the robot is to acquireenough information about the environment so that it can, even inthat case move, from the starting location to the target position.In this paper, we build upon our previously described multiagentsystem for outdoor landmark-based navigation [2]. It is composedof three systems: the Pilot, responsible for all motions of the robot,the Vision system, responsible for identifying and tracking land-marks and for detecting obstacles, and the Navigation system, re-sponsible for choosing high-level robot motions.These three systems must cooperate to achieve the overall task ofreaching the target. For instance, the Pilot needs the Vision systemto identify obstacles and the Navigation system to select a path tothe goal. The systems are also competing, for instance, the Pilotand the Navigation system both compete for the Vision system. ThePilot needs it for obstacle avoidance, while the Navigation systemneeds it for landmark detection and tracking.To manage this cooperation and competition, in [2] we had chosena bidding mechanism. Each system generates bids for the servicesoffered by the Pilot and Vision systems. The service actually exe-cuted by each system depends on the winning bid at each point intime. In [2] we proposed bidding functions to obtain good perfor-mance from the combined system. In this paper we use Reinforce-ment Learning (RL) [5] to tune the parameters of those functions.The Navigation system is also implemented as a multiagent sys-tem composed of six agents with the following goals: keep the tar-get located with maximum precision and reach it (Target Tracker),keep the risk of losing the target low (Risk Manager), recover from