Unsupervised parsing is the task of inducing syntactic tree structures (constituency or dependency) from unannotated text. Unlike supervised parsing, which learns from treebanks of annotated sentences, unsupervised parsing must discover syntactic structure from the distributional patterns in raw text alone. This is arguably one of the hardest problems in NLP, as syntactic structure is only indirectly reflected in surface word sequences. Progress has been slow but steady, with recent neural approaches achieving significantly better results than classical methods.
Unsupervised Constituency Parsing
Unlabeled bracketing F1 (UF1): fraction of correct span brackets
(ignoring constituent labels, which are not meaningful without supervision)
Right-branching baseline: ~39% UF1 on English PTB
Left-branching baseline: ~9% UF1 on English PTB
CCL (Seginer, 2007): ~71% UF1
Neural PCFG models: ~55–65% UF1
PRPN / Ordered Neurons: ~47–55% UF1
Unsupervised constituency parsing has been approached through both grammar-based and neural methods. Grammar-based approaches include the inside-outside algorithm for PCFGs (Baker, 1979), constituent-context models (Klein and Manning, 2002), and Bayesian models with structural priors. A key finding is that naive EM on PCFGs does not recover meaningful structure; strong inductive biases are needed. The CCM (Constituent-Context Model) of Klein and Manning (2002) was a breakthrough, achieving ~71% F1 by modeling the co-occurrence of yield strings and surrounding contexts.
Unsupervised Dependency Parsing
Unsupervised dependency parsing has seen notable progress through the Dependency Model with Valence (DMV) of Klein and Manning (2004), which models whether a head has already generated dependents in each direction. The DMV, trained with EM, was the first model to outperform a left-chain baseline on English. Subsequent work improved results through better initialization (using POS tag clustering), structural annealing, and Bayesian priors. Neural extensions parameterize the DMV with neural networks for better generalization.
Challenges and Prospects
Unsupervised parsing remains far below supervised methods in accuracy. The fundamental challenge is that there are many possible tree structures consistent with the surface statistics of text, and the linguistically correct one is not always the most statistically salient. Evaluation itself is problematic: the unlabeled bracketing metric may not capture all aspects of the structures these models learn, and there is debate about whether unsupervised models should be evaluated against linguist-designed conventions. Despite these challenges, unsupervised parsing provides insights into the information content of raw text and the inductive biases needed for language acquisition.