We therefore employ semi-supervised learning, and use a pre-training procedure to learn the parameters of the encoder.
we train the encoder as part of a neural dialogue model; we attach a third decoder RNN that takes the output of the encoder as input, and train it to predict the next utterance of a dialogue conditioned on the context.