论文信息 - Text Mining and Its Applications

Text Mining and Its Applications

As nowadays data centers are processing more jobs and collecting more data, the system status monitoring and analyzing functionality ensuring the availability, scalability and efficiency becomes more and more important. In order to build an automated status monitoring and alerting system, we need to group jobs performed at a data center upon jobs’ characteristics. Since the job names are generated by system users at will, it is very hard to group them in order to monitor the job status efficiently. Thus we need to find some methods to sort out the system log, and help to group jobs that are beneficial for improving the accuracy and efficiency of the system analysis. This paper proposes a text mining algorithm and its application in grouping jobs for log analysis. Keywords—Information Retrieval; Log Analysis; Text Mining; TF-IDF

Buyang Cao | Shengyu Guo

[1] Ravinder Kaur,et al. Hadoop: Addressing challenges of Big Data , 2014, 2014 IEEE International Advance Computing Conference (IACC).

[2] Karen Spärck Jones. A statistical interpretation of term specificity and its application in retrieval , 2021, J. Documentation.

[3] Hinrich Schütze,et al. Scoring , term weighting and thevector space model , 2015 .

[4] Dik Lun Lee,et al. Document Ranking and the Vector-Space Model , 1997, IEEE Softw..

[5] Michele Colajanni,et al. Detecting Behavioral Variations in System Resources of Large Data Centers , 2011, 2011 IEEE 11th International Conference on Computer and Information Technology.

[6] Karen Sparck Jones. A statistical interpretation of term specificity and its application in retrieval , 1972 .

[7] Akiko Aizawa,et al. An information-theoretic perspective of tf-idf measures , 2003, Inf. Process. Manag..

[8] Hinrich Schütze,et al. Introduction to Information Retrieval: Scoring, term weighting, and the vector space model , 2008 .

[9] Himabindu Pucha,et al. Towards Optimizing Hadoop Provisioning in the Cloud , 2009, HotCloud.

[10] Gerard Salton,et al. A vector space model for automatic indexing , 1975, CACM.

[11] Qiang Fu,et al. Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis , 2009, 2009 Ninth IEEE International Conference on Data Mining.