论文信息 - Using conservative estimation for conditional probability instead of ignoring infrequent case

Using conservative estimation for conditional probability instead of ignoring infrequent case

There are several estimators of conditional probability from observed frequencies of features. In this paper, we propose using the lower limit of confidence interval on posterior distribution determined by the observed frequencies to ascertain conditional probability. In our experiments, this method outperformed other popular estimators.

[1] William A. Gale,et al. Good-Turing Frequency Estimation Without Tears , 1995, J. Quant. Linguistics.

[2] E. S. Pearson,et al. THE USE OF CONFIDENCE OR FIDUCIAL LIMITS ILLUSTRATED IN THE CASE OF THE BINOMIAL , 1934 .

[3] Emmanuel Morin,et al. A Comparison of Smoothing Techniques for Bilingual Lexicon Extraction from Comparable Corpora , 2013, BUCC@ACL.

[4] Rakesh Agarwal,et al. Fast Algorithms for Mining Association Rules , 1994, VLDB 1994.

[5] Kenneth Ward Church,et al. Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[6] Tobias Scheffer. Finding association rules that trade support optimally against confidence , 2005 .

[7] Rafael Berlanga,et al. Knowledge based word-concept model estimation and refinement for biomedical text mining. , 2015, Journal of biomedical informatics.

[8] F ChenStanley,et al. An Empirical Study of Smoothing Techniques for Language Modeling , 1996, ACL.

[9] I. Good. THE POPULATION FREQUENCIES OF SPECIES AND THE ESTIMATION OF POPULATION PARAMETERS , 1953 .