FlyingHand: extending the range of haptic feedback on virtual hand using drone-based object recognition

This paper presents a Head Mounted Display (HMD) integrated system, that uses a drone and a virtual hand to help the users explore remote environment. The system allows the users to use hand gestures to control the drone and identify the Objects of Interest (OOI) through tactile feedback. The system uses a Convolutional Neural Network to perform object classification with the drone captured image and provides a virtual hand to realize interaction with the object. Accodingly, tactile feedback is also provided to users' hands to enhance the virtual hand body ownership. The system aims to help users assess space and objects regardless of body limitations, which could not only benefit elderly or handicapped people, but make potential contributions in environment measurement and daily life as well.