In this paper, we propose the Extended Transformer model for Abstractive Document Summarization (ETADS) to tackle the above issues.
we design a novel focusattention mechanism and saliency-selection network equipped in the encoder and decoder respectively: (1) To comprehensively encode the documents, we design a focus-attention mechanism, where a learnable Gaussian focal bias is employed as a regularization term on attention scores.