Recently, large-scale pretrained language models (LM) such as BERT (Devlin et al., 2019) and GPT-2 (Radford et al., 2019) are used to improve the performance of dialog models, however in the cost of tens-fold larger model sizes and computations.
remarkably, LABES-S2S is optimized under the principled variational learning framework.