7: Therefore, the final sentence representation S T is formally represented as: Compared with the RPEHead model, the MPRHead model applies the multi-head mechanism to the RPEs.
rPEs are encoded into the sentence representation from different vector sub-spaces, as shown in Fig 1(c).