In addition, on higher levels, the directions from other words to each of these three tokens will also be identical.
the use of "directions" 4 allows the network to represent entities as a single embedding in a fully differentiable fashion, whilst keeping the sequence length constant.