Our attention mechanism does not employ non-linear transformations.
the attentive context ([ − → C ; ← − G ]) does not significantly change its colour from the context word encoding ( − → C and ← − G ).