Both of its encoder and decoder consist of 6-layer Transformers, where the inner dimension of feed-forward networks (FFN), hidden state size, and the number of attention head are set to 4096, 1024, and 16.
we implement Back-Translation based on the source code6 to generate a new question data for each original question; (3) Text-VAE (Bowman et al., 2016;Liu et al., 2019a): it uses RNNbased VAE to generate a new question data for each question of SQuAD 2.0.