Recently, various interaction was demanded from rapid development and use of portable device. So, operation of touch interface in tablet and smart phone needs many requirements and proactivity. In this paper, for variety of users and convenient interaction, It is aimed to realize virtual reality mouse of portable device. It reflected similar interaction of an existing mouse and it used infrared image without intensity of illumination. Also, It analyzed finger point information of middle and index finger of users and it designed virtual mouse without proactivity and constraint of space.