This query chain can be decomposed into two parts: a key represented by i k and a query represented by The former is modulated through the weight matrix W k , and tightly associated with the corresponding input token.
information carried by the key remains intact during the evolution of time step t. the latter, induced by the weight matrix W q , highly depends on the position and length of this chain, which dynamically changes between different token pairs.