Utterance Encoder: Each utterance in the dialog is processed sequentially using single layer Bidirectional Long Short Term Memory (BiLSTM) (Dyer et al., 2015) network and self-attention mechanism (Vaswani et al., 2017) to get the utterance representation.
given an utterance with one-hot encoding for the tokens, u k = {w k,1 , w k,2 , ...., w k,L }, each token is mapped to a vector v k,i = Ew k,i ; i = 1, 2, ...L using pretrained embeddings (matrix E).