Specifically, these work instantiate their encoder-decoder framework by choosing recurrent neural networks (Cheng and Lapata, 2016;Nallapati et al., 2017;Zhou et al., 2018) as encoder, auto-regressive decoder (Chen and Bansal, 2018;Jadhav and Rajan, 2018;Zhou et al., 2018) or non auto-regressive decoder (Isonuma et al., 2017;Narayan et al., 2018;Arumae and Liu, 2018) as decoder, based on pre-trained word representations (Mikolov et al., 2013;Pennington et al., 2014).
how to use Transformer in extractive summarization is still a missing issue.