To be consistent with the seq2seq model for AMR parsing, the pre-trained models in this paper are all built on the Transformer.
for each pretraining task listed in Table 1, we learn a seq2seq model which will be used to initialize seq2seq model for AMR parsing in the fine-tuning phase.