We complement the state with an attention over the input sentence (Bahdanau et al., 2014).
(3) Incorporating contextualized vectors (with BERT) and compare their effectiveness with detailed ablation experiments.