Many statistical methods have been adapted for terminology recognition to improve its accuracy. However, since previous studies have been carried out in a single core or a single machine, they have difficulties in real-time analysing explosively increasing documents. In this study, the task where bottlenecks occur in the process of terminology recognition is classified into linguistic processing in the process of `candidate terminology extraction` and collection of statistical information in the process of `terminology weight assignment`. A terminology recognition system is implemented and experimented to address each task by means of the distributed parallel processing-based MapReduce. The experiments were performed in two ways; the first experiment result revealed that distributed parallel processing by means of 12 nodes improves processing speed by 11.27 times as compared to the case of using a single machine and the second experiment was carried out on 1) default environment, 2) multiple reducers, 3) combiner, and 4) the combination of 2)and 3), and the use of 3) showed the best performance. Our terminology recognition system contributes to speed up knowledge extraction of large scale science and technology documents.
[1]
Hideki Mima,et al.
Automatic recognition of multi-word terms:. the C-value/NC-value method
,
2000,
International Journal on Digital Libraries.
[2]
Kenneth Ward Church,et al.
Word Association Norms, Mutual Information, and Lexicography
,
1989,
ACL.
[3]
Vasileios Hatzivassiloglou,et al.
Translating Collocations for Bilingual Lexicons: A Statistical Approach
,
1996,
CL.
[4]
Won-Kyung Sung,et al.
Multi-words Terminology Recognition Using Web Search
,
2011,
FGIT-UNESST.
[5]
Paul M. B. Vitányi,et al.
The Google Similarity Distance
,
2004,
IEEE Transactions on Knowledge and Data Engineering.
[6]
Slava M. Katz,et al.
Technical terminology: some linguistic properties and an algorithm for identification in text
,
1995,
Natural Language Engineering.