Sport event detection is an important task in the research area of human behavior recognition. Owing to different motion models of different sport events, existing general human pose recognition methods cannot achieve high accuracy for sport events detection and counting. In this paper, we propose and implement a sport event detection and counting algorithm framework based on human skeletal information. Experimental evaluation results demonstrate that the algorithm can accurately detect the sit-up events and count the number of sit-ups with the highest average accuracy of 96%.