Computer vision’s demand in applications like visual surveillance, video retrieval and human-computer interaction are increasing day by day. Human action recognition is one such vital research area under computer vision. Many datasets dedicated to human action recognition have been created and are accompanied with equally large number of techniques of recognition in recent years. In this paper, we have proposed a technique to recognize two-person interactions based on a benchmark dataset from University of Texas, Austin called UT interaction Dataset. The task is challenging due to variations in motion, dynamic background, recording settings and inter-personal differences. Our technique includes concatenated Motion History Images (cMHI). The produced cMHI templates are incorporated with Local Binary Pattern (LBP) and Histogram of Oriented Gradient (HOG) to develop smart feature vector for each class. Finally, Support Vector Machine (SVM) is exploited to classify various actions.