In order to solve the balance problems between efficiency and accuracy encountered in content-based video retrieval, in this paper, we propose firstly a video retrieval scheme based on coarse granularity. To achieve the coarse-grained video retrieval, we use inverted index. Further, we expose an improved BLAST algorithm to realize fine-grained retrieval of video fingerprint. Finally, simulation results show the good efficiency and accuracy of the algorithm we have proposed. Therefore, this algorithm provides effective technical support for the security regulation and quick retrieval of Internet video.