Addressing Scalability Issues of Named Entity Recognition Using Multi-Class Support Vector Machines

This paper explores the scalability issues associated with solving the Named Entity Recognition (NER) problem using Support Vector Machines (SVM) and high-dimensional features. The performance results of a set of experiments conducted using binary and multi-class SVM with increasing training data sizes are examined. The NER domain chosen for these experiments is the biomedical publications domain, especially selected due to its importance and inherent challenges. A simple machine learning approach is used that eliminates prior language knowledge such as part-of-speech or noun phrase tagging thereby allowing for its applicability across languages. No domain-specific knowledge is included. The accuracy measures achieved are comparable to those obtained using more complex approaches, which constitutes a motivation to investigate ways to improve the scalability of multi- class SVM in order to make the solution more practical and useable. Improving training time of multi-class SVM would make support vector machines a more viable and practical machine learning solution for real-world problems with large datasets. An initial prototype results in great improvement of the training time at the expense of memory requirements. In previous work (11), a simple architecture that eliminates language and domain-specific knowledge from the named entity recognition process is applied to the English biomedical entity recognition task, as a baseline for other languages and domains. The biomedical field NER remains a challenging task due to growing nomenclature, ambiguity in the left boundary of entities caused by descriptive naming, difficulty of manually annotating large sets of training data, strong overlap among different entities, to cite a few of the NER challenges in this domain. The approach used reduces the pre- and post-processing of the textual data to a minimum and capitalizes on SVM's strong generalization ability to classify the named entities. The accuracy measures achieved are comparable to those obtained using more complex techniques, which encourage us to explore ways to improve the scalability of multi-class support vector machines. In this paper, the results of a set of scalability experiments are reported. These experiments use binary and multi-class SVM with a large set of real-world data from the biomedical literature. In Section II, the theory of binary and multi-class support vector machines is briefly introduced. Section III describes the experiments' design and summarizes the results of a baseline experiment conducted during the previous work (11) in order to assess the feasibility of our language and domain- independent machine learning NER approach using SVM and high-dimensional features. The baseline experiment design reduces pre-processing to feature extraction and eliminates the use of prior language or domain knowledge. The results of the baseline experiment are a motivation to explore ways to address the scalability issues of the All-Together multi-class SVM approach. Improving scalability of multi-class SVM would provide the research community with a practical and powerful machine learning solution for named entity recognition that promotes the use of high-dimensional features in place of more complex labor and time expensive pre- and post-processing tasks, and simplifies the NER process while achieving good accuracy and performance measures. In Section IV, the results of several sets of single-class and multi-class scalability tests using SVM and increasing training data size are reported and their impact on training time is examined. A sample of preliminary results using a prototype multi-class implementation based on SVM-Perf is also presented.