When one sentence learns attention to another sentence, the attention is performed between two parallel layers and oriented to the intermediate representations from the preceding layer of another one.
given the representations of P and Q: , each cross attention P(t)→Q will use the original semantics H 0 Q of Q for interaction, where t = {1, • • • , T} and t = 1 represents P using the original representation H 0 P .