Projection Based Virtual Tablets System Involving Robust Tracking of Rectangular Objects and Hands

We propose a novel projection based markerless AR system that realizes multiple virtual tablets. This system detects position and posture of any rectangular objects, projects GUI to these objects, and detects touch gesture on objects. As a result, by using this system, we do not need any smart devices, but only a mere rectangular object, e.g. tissue box, book, cushion, table and so on. It does not really matter whether the tablet computer is in the living room for browsing the internet, for playing games, or for controlling consumer devices e.g. TV and air-conditioner. In order to realize this system, we developed a novel algorithm to detect arbitrary rectangular objects. This can recognize position and posture of rectangular object without markers. We measured error in the case of overlapping, as a result, experimental result shows our algorithm is more robust than existing algorithms.