The sub-layer also takes permutation-agnostic topicsensitive sentence vectors obtained from our encoder as the key and value matrices K and V. We follow Transformer for the remaining coherent topical context matching process, including multihead attention strategy.
with the multiple attention modules, TGCM draws global dependencies among sentences by attending over the topic-sensitive sentence vectors repeatedly.