Document Clustering Algorithm Based on Dynamic Interval Mapping

Archival storage is becoming a research hotspot with information digitization accelerating,where space utilization and scalability are very important.Using content-based chunking storage to achieve data deduplication is an effective way to improve storage space utilization,however,it is inefficiency to find shared chunks in all of the huge scale of archival data.We introduced the thought of dynamic interval mapping to information clustering,and presented the DC-DIM(Document Clustering algorithm based on Dynamic Interval Mapping).The algorithm uses chunking and feature extraction methods to generate the feature-set of document,and map it on interval links,then choose the document’s storage container according to its feature-set’s distribution on interval links.By this way,those documents with high similarity(shared a lot of contents) will be clustered,then,it will be very convenient to improve the space utilization and data management.