Mobile Magic Hand: Camera Phone Based Interaction Using Visual Code and Optical Flow

We propose the "Mobile Magic Hand" interface; it is an extension of our previous visual code-based interface system. Once the user acquires the visual code of interest, the user can then manipulate the related virtual object/system without having to keep the camera centered on the visual code. Our new interface does this analyzing the optical flow as captured by the camera. For example, consider a visual code that represents a 3D object, such as a dial. After selecting the code, the user can freely rotate and/or move the virtual object without having to keep the camera pointed at the code. This interface is much more user friendly and is more intuitive since the user's hand gestures can be more relaxed, more natural, and more extensive. In this paper, we describe "Mobile Magic Hand", some applications, and a preliminary user study of a prototype system.