Baby Steps: How “Less is More” in Unsupervised Dependency Parsing

We present an empirical study of two very simple approaches t o unsupervised grammar induction. Both are based on Klein and Manning’s Dep endency Model with Valence. The first, Baby Steps, requires no initialization and bootstraps itself via iterated learning of increasingly longer sentences. Th is method substantially exceeds Klein and Manning’s published numbers and achieves 39.4% accuracy on Section 23 of the Wall Street Journal corpus — a result that is lready competitive with the recent state-of-the-art. The second, Less is More, is based on the observation that there is sometimes a trade-off between the quant ity and complexity of training data. Using the standard linguistically-infor med prior but training at the “sweet spot” — sentences up to length 15, it attains 44.1% accuracy, beating state-of-the-art. Both results generalize to the Brown cor pus and shed light on opportunities in the present state of unsupervised depende ncy parsing.