Interaction on Augmented Reality with Finger Detection and Hand Movement Recognition

Indonesia’s rare animals are animals whose habitats exist only in Indonesia and are endangered by IUCN (International Union for Conservation of Nature and Natural Resources). In this study, we applied Augmented Reality (AR) technology to represent the endangered species in Indonesia combined with image processing techniques as the interaction between users and the virtual objects of endangered animals. The initial stage is the separation of the desired object from the background with the help of HSV colour method, as well as detecting contour with contour detector. As for the calculation of the number of fingers we are using convex hull and convexity defects. The results of this stage is the number of fingers that will be used as a reference selection of endangered animal. These series of processes generates an augmented reality application where users can view and provide instructions with virtual objects of endangered species of Indonesia. This can provide a different experience for users to learn about Indonesia’s endangered species.