The position embedding is also used to compute the context vector c k : The discourse state and attention over previous utterances allow the model to consider the interaction history when generating queries.
we observe that context-dependent reasoning often requires generating sequences that were generated in previous turns.