Irregularity analysis in time series data

Government and corporations nowadays collect time series data at lowest possible details such as by locations, parts, products, or even individuals. Most of data cleaning methods assume one known type of irregularity. This paper provide a framework for the situation that there are multiple irregularities hiding in large volumes of cross sectional time series and develops a data mining platform to capture these key irregularities one by one based on their importance. It attempts to automate how a data analyst looking at time series graphs when cleaning the data (but there are too many to look at). Clustering is applied to group time series with similar pattern, and the principal irregular component of the dominated time series group is extracted and adjusted. The platform continues to cluster, extract and adjust the next significant irregular components iteratively. Finally all these significant irregular components are summarized in graphic forms to help analysts to know the data better and faster before any analysis and modeling.