With this motivation, we apply a conditional random field (CRF) (Lafferty et al., 2001) on top of the word-level BiLSTM to perform POS tag inference with global normalization, addressing the "label bias" problem.
given an input sentence, we pass the output sequence of the word-level BiLSTM to a firstorder chain CRF to compute the conditional probability of the target label sequence: where θ represents all of the model parameters (in the BiLSTMs and CRF), s and y denote the input embeddings and the target POS tag sequence, respectively, for the given sentence.