Conditional Random Fields (CRF) (Lafferty et al., 2001) is an effective method to sequence labeling problem and has been widely used in many NLP tasks (Sutton and McCallum, 2012).
we propose a restricted self-attention mechanism, which only collects information from a fixed neighborhood.