Repeating this mechanism in consecutive layers allows for information to flow over long distances.
for each input token, each attention head scales linearly in memory and time in the context size, or attention span.