We build on the QANet (Yu et al., 2018) model by modifying the encoder of its self-attention modeling layer.
we specialize self-attention heads to focus on specific discourse-semantic annotations, such as, e.g., an ARG1 relation in SRL, a CAUSA-TION relation holding between clauses in shallow discourse parsing, or coreference relations holding between entity mentions.