Inference Algorithms for Pattern-Based CRFs on Sequence Data

We consider Conditional random fields (CRFs) with pattern-based potentials defined on a chain. In this model the energy of a string (labeling) $$x_1\ldots x_n$$x1…xn is the sum of terms over intervals [i, j] where each term is non-zero only if the substring $$x_i\ldots x_j$$xi…xj equals a prespecified pattern w. Such CRFs can be naturally applied to many sequence tagging problems. We present efficient algorithms for the three standard inference tasks in a CRF, namely computing (i) the partition function, (ii) marginals, and (iii) computing the MAP. Their complexities are respectively $$O(\textit{nL})$$O(nL), $$O(\textit{nL} \ell _{\max })$$O(nLℓmax) and $$O(\textit{nL} \min \{|D|,\log (\ell _{\max }\!+\!1)\})$$O(nLmin{|D|,log(ℓmax+1)}) where L is the combined length of input patterns, $$\ell _{\max }$$ℓmax is the maximum length of a pattern, and D is the input alphabet. This improves on the previous algorithms of Ye et al. (NIPS, 2009) whose complexities are respectively $$O(\textit{nL} |D|)$$O(nL|D|), $$O\left( n |\varGamma | L^2 \ell _{\max }^2\right) $$On|Γ|L2ℓmax2 and $$O(\textit{nL} |D|)$$O(nL|D|), where $$|\varGamma |$$|Γ| is the number of input patterns. In addition, we give an efficient algorithm for sampling, and revisit the case of MAP with non-positive weights.