The size of attention layer is set to 500.
the model is trained by three steps: 1) training word vectors and two language models, 2) training seq2seq conversation model with pretrained LMs, and 3) fine-tuning task-specific classification layers, after removing softmax of the conversation model.