Soccer Video Structure Analysis by Parallel Feature Fusion Network and Hidden-to-Observable Transferring Markov Model

Automated analysis of broadcast soccer game video is a challenging computer vision problem. Prior to performing high-level analysis (such as event detection), accurate classification of shot views and play–break segmentation are required to analyze the structure of soccer video. A novel deep network called parallel feature fusion network (PFF-Net) combines local and full-scene features to produce accurate shot view classification based on camera zoom and out-of-field status. Then, a novel hidden-to-observable Markov model (H2O-MM) is introduced to determine play/break status of the shots. Testing is performed using a variety of professional broadcast soccer videos. Variations of the PFF-Net are considered and compared with four existing methods where the PFF-Net demonstrates superior performance (92.6%). The H2O-MM has the accuracy of 98.7% for play–break segmentation, which is an improvement over two existing hidden Markov models. The new methods provide improved temporal labeling of broadcast soccer videos, which can be used to further improve overall automated event detection.