Our key insight is that we can use syntactic information as a proxy to the learned discrete latent space of the LT.
to retain information about the order of the input words, the transformer also includes positional encodings, which are added to the source word embeddings.