论文信息 - Research on Automatic Text Summarization Method Based on TF-IDF

Research on Automatic Text Summarization Method Based on TF-IDF

In order to quickly obtain the main information contained in news documents, reduce redundant information and improve the efficiency of finding news with specific content. A Chinese text summarization method based on TF-IDF is proposed. This method uses TF-IDF to calculate the importance of each word in the article, and calculates the TF-IDF of each sentence based on the TF-IDF value of the word. In order to avoid the effect of sentence length on the calculation of sentence TF-IDF value. The sliding window is used to calculate the mean of all words TF-IDF in each sliding window using the given sliding window size. Use the value of the sliding window with the largest mean value in each sentence as the TF-IDF value of the sentence. Combined with other feature of the sentence, the importance of the sentence is calculated. The sentences with the specified length or number of words are intercepted and arranged according to the order of the articles to form a summarization of the article. After comparison experiments, the method is superior to the text summarization scheme based on TextRank method in terms of efficiency and effect.

Tao Zhang | Cai Chen

[1] Li-Ping Jing,et al. Improved feature selection approach TFIDF in text mining , 2002, Proceedings. International Conference on Machine Learning and Cybernetics.

[2] Stefano Ferilli,et al. An Abstract Argumentation-Based Approach to Automatic Extractive Text Summarization , 2018, IRCDL.

[3] Jackie Chi Kit Cheung,et al. What comes next? Extractive summarization by next-sentence prediction , 2019, ArXiv.

[4] Jugal K. Kalita,et al. Exploring Sentence Vector Spaces through Automatic Summarization , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[5] Dan Lim,et al. Convolutional Attention-based Seq2Seq Neural Network for End-to-End ASR , 2017, ArXiv.

[6] Yong Zhang,et al. A Hierarchical Attention Seq2seq Model with CopyNet for Text Summarization , 2018, 2018 International Conference on Robots & Intelligent System (ICRIS).

[7] Ong Sing Goh,et al. Automatic Text Summarization: Soft Computing Based Approaches , 2018 .

[8] Akiko Aizawa. The feature quantity: an information theoretic perspective of Tfidf-like measures , 2000, SIGIR '00.