Visual exploration of large data sets

DEVise is a data visualization and exploration system capable of handling large data sets using off-the-shelf hardware with minimal memory requirements. Data can be large in volume, complex in structure (multi-dimensional and/or hierarchical), and may be imported from different sources such as database servers, external programs, and World Wide Web resources. Commercial and scientific databases can also be linked to the DEVise to allow the user to visualize and analyze related information from heterogeneous sources. Associations between data sources are developed interactively as the user gains more knowledge of the data being explored. To assist in handling large data sets, DEVise allows a user to logically split the data into more manageable units at different levels. The user selects a data source, a data stream within a data source (e.g. a time series), attributes of a stream, and a mapping of attributes to graphical objects. At each step, the selections made by the user reduce the data volume. DEVise takes advantage of this form of 'data compression' to optimize its caching strategies and to minimize the accesses needed to fetch data from tertiary storage, for example. DEVise supports users with different expertise levels by automating most tasks performed by a novice user and by also providing a programming interface that allows new data sources to be defined, new graphical objects to be used, and custom storage policies to be employed.