Markov logic for machine reading

A long-standing goal of AI and natural language processing (NLP) is to harness human knowledge by automatically understanding text. Known as machine reading, it has become increasingly urgent with the rise of billions of web documents. However, progress in machine reading has been difficult, due to the combination of several key challenges: the complexity and uncertainty in representing and reasoning with knowledge, and the prohibitive cost in providing direct supervision (e.g., designing the meaning representation and labeling examples) for training a machine reading system. In this dissertation, I propose a unifying approach for machine reading based on Markov logic. Markov logic defines a probabilistic model by weighted first-order logical formulas. It provides an ideal language for representing and reasoning with complex, probabilistic knowledge, and opens up new avenues for leveraging indirect supervision via joint inference, where the labels of some objects can be used to predict the labels of others. I will demonstrate the promise of this approach by presenting a series of works that applied Markov logic to increasingly challenging problems in machine reading. First, I will describe a joint approach for citation information extraction that combines information among different citations and processing stages. Using Markov logic as a representation language and the generic learning and inference algorithms available for it, our solution largely reduced to writing appropriate logical formulas and was able to achieve state-of-the-art accuracy with substantially less engineering effort compared to previous approaches. Next, I will describe an unsupervised coreference resolution system that builds on Markov logic to incorporate prior knowledge and conduct large-scale joint inference. This helps compensate for the lack of labeled examples, and our unsupervised system often ties or even outperforms previous state-of-the-art supervised systems. Finally, I will describe the USP system, the first unsupervised approach for jointly inducing a meaning representation and extracting detailed meanings from text. To resolve linguistic variations for the same meaning, USP recursively clusters expressions that are composed with or by similar expressions. USP can also induce ontological relations by creating abstractions to assimilate commonalities among non-synonymous meaning clusters. This results in a state-of-the-art end-to-end machine reading system that can read text, extract knowledge and answer questions, all without any labeled examples. Markov logic provides an extremely compact representation of the USP model, and enables future work to "close the loop" by incorporating the extracted knowledge into the model to aid further extraction.