论文信息 - The Design, Implementation, and Use of the Ngram Statistics Package

The Design, Implementation, and Use of the Ngram Statistics Package

The Ngram Statistics Package (NSP) is a flexible and easy-to-use software tool that supports the identification and analysis of Ngrams, sequences of N tokens in online text. We have designed and implemented NSP to be easy to customize to particular problems and yet remain general enough to serve a broad range of needs. This paper provides an introduction to NSP while raising some general issues in Ngram analysis, and summarizes several applications where NSP has been successfully employed. NSP is written in Perl and is freely available under the GNU Public License.

Satanjeev Banerjee | Ted Pedersen | S. Banerjee | T. Pedersen

[1] Kenneth Ward Church,et al. Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[2] Philip Resnik,et al. Word-level Alignment for Multilingual Resource Acquisition , 2002 .

[3] Ted Pedersen,et al. Fishing for Exactness , 1996, ArXiv.

[4] Ted Pedersen. Machine Learning with Lexical Features: The Duluth Approach to SENSEVAL-2 , 2001, SENSEVAL@ACL.

[5] Alfa-Informatica,et al. Corpus-based acquisition of collocational prepositional phrases , 2022 .

[6] Graeme Hirst,et al. Acquiring Collocations for Lexical Choice between Near-Synonyms , 2002, Proceedings of the ACL-02 workshop on Unsupervised lexical acquisition -.

[7] Ted Pedersen,et al. Significant Lexical Relationships , 1996, AAAI/IAAI, Vol. 1.

[8] Claude E. Shannon,et al. Prediction and Entropy of Printed English , 1951 .

[9] Alastair J. Gill,et al. Taking Care of the Linguistic Features of Extraversion , 2019, Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science Society.

[10] Kenneth Ward Church,et al. Using Suffix Arrays to Compute Term Frequency and Document Frequency for All Substrings in a Corpus , 2001, Computational Linguistics.

[11] Ted Dunning,et al. Accurate Methods for the Statistics of Surprise and Coincidence , 1993, CL.

[12] Ted Pedersen,et al. A Decision Tree of Bigrams is an Accurate Predictor of Word Sense , 2001, NAACL.

[13] A. van der Wouden. Collocational behaviour in non content words , 2001 .