Conditional Random Fields with Decode-based Learning: Simpler and Faster

Conditional random fields (CRF) is one of the most famous approaches for structured classification. It is a structured gradient-based method, which has high accuracy but with drawbacks: very slow training, hard to implement in the tasks with complex structures, and no support of decode-based optimization (which is important in many cases). To address these issues, we propose a simple and fast solution, a decode-based probabilistic online learning method, called CRF with decode-based learning (DBL-CRF). The proposed DBL-CRF decodes the output candidates, derives probabilities, and conduct efficient online learning. The method has the similar probabilistic information as CRF, but supports decode-based optimization and does not need gradient computation. We show that this method is with fast training, very simple to implement, with top accuracy, and with theoretical guarantees of convergence. Experiments on well-known tasks show that our method has better accuracy and much faster speed than our strong baseline CRF systems. The code is available at https://github.com/lancopku/Decode-CRF