Recognition and visual feature matching of text region in video for conceptual indexing

An indexing method for content-based image retrieval by using textual information in video is proposed. Indices extracted from textual information make it possible to retrieve video data by a conceptual query, such as a topic or a person's name, and organize flat video data into structured video data based on its conceptual content. To this end, we developed a text extraction and recognition algorithm and a visual feature matching algorithm for indexing and organizing video data at a conceptual level. The text extraction and recognition algorithm identifies frames in the video which contain text, extracts the text regions from the frame, finds text lines, and recognizes characters in the text line. The visual feature matching algorithm measures the similarity of frames containing text of find frames with similar appearances text, which can be considered topic change frames. Experiments using real video data showed that our algorithm can index textual information reliably and that it has good potential as a tool for making content-based conceptual-level queries to video databases.