Multi-Databases: Removal of Redundant Information

Abstract : This effort included the partial development of a search engines for multimedia web documents and the complete implementation of a prototype methodology for removing (partially or totally) redundant information from multiple documents in an effort to synthesize new documents. A typical multimedia document contains free text and images and additionally has associating well-structured data. An SQL-like query language, WebSSQL, has been used to retrieve these types of documents. The main differences between Web SSQL and other proposed SQL extensions for retrieving web documents are that Web SSQL is similarity-based and supports conditions on images. This report also describes a software methodology for the detection and removal of redundant information (text paragraphs and images) from multiple retrieved documents. Documents reporting the same or related events and stories may contain substantial redundant information. The removal of the redundant information and the synthesis of these documents into a single document can not only save a user's time to acquire the information, but also storage space to archive the data. The methodology reported here consists of techniques for analyzing text paragraphs and images as well as a set of similarity criteria used to detect redundant paragraphs and images. The methodology developed in this project has the ability either to work independently with text paragraphs and images, or to combine both in one synthetic document.