A real-time HCI system based on infrared tracking

In this paper, a novel Human-Computer Interaction (HCI) system based on IR tracking technology is proposed for big projection screen real-time HCI applications. The proposed system has been successfully applied in a multiple-user shooting game. In this system, each user holds an interactive device, which is based on an ADSP-BF533 hardware platform, to interact with the shooting game on the big projection screen. IR tracking technology is introduced and theories for projective transformation are studied in detail. A fast IR marker identification algorithm is developed for real-time interaction. Two groups of IR markers are adopted in the proposed system for IR tracking, which are fixed on the border of the big projection screen. To utilize the theories for projective transformation, each group of IR marker is composed of 4 IR LEDs with a rectangular arrangement. There is a CMOS sensor inside each interactive device to capture the image of IR markers. The captured image is then used for image processing and IR marker identification on the ADSP-BF533 hardware platform. According to the theories for projective transformation, the target position on the big projection screen of each interactive device can be obtained after the projective transformation from the CMOS image to the big projection screen. An experimental environment is set up and experiments are implemented to test the validation of the performance of the IR marker identification algorithm, i.e. tracking accuracy, time consuming and effective region of the proposed system. Experimental results show that the proposed system is very efficient and robust for big projection screen interaction application. The IR markers can be correctly identified with the identification algorithm developed in this paper. Experimental results also show that the developed system can achieve real-time processing and work with high tracking accuracy. The proposed system works well in a large effective region that allows multiple-user interaction. Moreover, the proposed system can be extended to bigger projection screen interaction applications with the adoption of more groups of IR marker.