Moreover, to further strengthen the global information, we implement self-attention (Vaswani et al., 2017) to mine the relationship of the annotation at a certain time step with other annotations.
the gated unit is able to find out both common n-gram features and global correlation.