Contrastive estimation

Citations Over TimeTop 1% of 2005 papers

Abstract

Conditional random fields (Lafferty et al., 2001) are quite effective at sequence labeling tasks like shallow parsing (Sha and Pereira, 2003) and named-entity extraction (McCallum and Li, 2003). CRFs are log-linear, allowing the incorporation of arbitrary features into the model. To train on unlabeled data, we require unsupervised estimation methods for log-linear models; few exist. We describe a novel approach, contrastive estimation. We show that the new technique can be intuitively understood as exploiting implicit negative evidence and is computationally efficient. Applied to a sequence labeling problem---POS tagging given a tagging dictionary and unlabeled text---contrastive estimation outperforms EM (with the same feature set), is more robust to degradations of the dictionary, and can largely recover by modeling additional features.

Related Papers

• → Recurrent conditional random field for language understanding(2014)112 cited
• → Hybrid semi-Markov CRF for Neural Sequence Labeling(2018)56 cited
• → Gradual transition detection with conditional random fields(2007)7 cited
• → Upgrading CRFS to JRFS and its Benefits to Sequence Modeling and Labeling(2020)5 cited
• → Hybrid semi-Markov CRF for Neural Sequence Labeling(2018)3 cited