Natural Language Processing: Overview

The advent of the World Wide Web has greatly increased demand for software tools and appliances for processing unstructured and semi-structured natural language text. Ancillary developments, such as corporate intranets, enterprise portals, and ubiquitous e-mail, have created many challenges and opportunities in application areas such as information retrieval, electronic commerce, and knowledge management. On the supply side, the development of language technology to address such attendant problems as information overload and rapid globalization has been facilitated by two technical breakthroughs. The first is conceptual, and represents a new emphasis upon empirical approaches to language processing that rely more heavily upon corpus statistics than linguistic theory. The second is computational, and consists of more powerful, networked machines that are capable of processing millions of documents and performing the billions of calculations that the statistical profiling of large corpora requires. This article outlines the new application areas and describes some of the advances that have been made. The emphasis is upon showing how the technical approaches outlined elsewhere in this encyclopedia can be combined to create products and services that have genuine value.