Therefore, absolute position (Vaswani et al., 2017) or relative position (Shaw et al., 2018) are generally used to capture the sequential order of words in the sentence.
we conduct probing tasks 3 (Conneau et al., 2018) to evaluate structure knowledge embedded in the encoder output in the variations of the Base model that are trained on En⇒De translation task.