Discarding early contextual utterances may cause information loss, but this loss is negligible because they only contribute the least amount of information .
the hierarchical self-attention follows two steps: 1) each utterance representation is computed using an utterance-level self-attention layer, and 2) a context representation is computed from M learned utterance representations using a context-level self-attention layer.