Enhancing data integration and discovery with ad hoc registries

The increased importance of public data in various fields has shifted the emphasis to the search-ability and discovery of data in a timely manner. Specialized data registries and meta data catalogs that index data, meta data and services exist for various fields and are continued to be maintained manually; this coupled with focused methods of searching and sorting meta data provide a powerful and effective mechanism for data discovery and data integration. However in many cases, these meta data registries fail to capture meta data and information on a large number of available web services and data sources, thus reducing the overall effectiveness of a meta data search. Rather than requiring data publishers to manually publish meta data registries, we propose a novel way of automating the creation of registries with basic meta data. Using a set of standard compliant catalogs, services and specifications and a list of data sources, we are able to populate catalogs with meta data elements, thus creating ad hoc registries that aid in data discovery, which can then be used by specialized client and methods for effective data discovery and integration.