We choose BERT (Devlin et al., 2018) as the backbone of the encoder.
the encoder can take advantage of pre-training, and the multi-layer bidirectional attention mechanism in BERT allows a dialogue context and the associated knowledge to sufficiently interact with each other, resulting in context-aware knowledge representations.