Projection-based Localization and Control Method of Robot Swarms for Swarm User Interfaces

The augmented reality systems, in which computer graphics and multiple mobile robots are cooperatively controlled, are useful for an intuitive understanding of complicated information because users can touch and operate information directly through robots [10, 18]. The swarm user interfaces using cooperatively controlled multiple mobile robots are also a useful approach [11]. These systems express various modalities utilizing the assembly and distribution of robots; however, they cannot realize a collaboration between computer graphics and robots. Therefore, the system, in which computer graphics and multiple mobile robots seamlessly cooperate, can be expected to expand the domain of the swarm user interfaces. Two problems remain in terms of ensuring a seamless collaboration between computer graphics and multiple mobile robots. First, previous methods use external measurement systems for localization that use the computer vision technology. However, the camera positions must be corrected and calibrated, and the spatial position of the robot in the camera images must be computed. Localization methods without computer vision may depend on a laser, sonar, or visible light communication [13]. However, these approaches have limited accuracy because of the resolution of each sensor. Augmented Coliseum [10] approaches this issue using a display-based measurement control system (DMCS) [19]. This technology eliminates the need for position-measuring devices and can support multiple robots on display. However, a prior initialization of tracking robots by marker-pattern images is necessary. Therefore, we cannot add or remove robots.