This indicates that the learned RPEs are beneficial for the Transformer system.
there is a new sequence of mixed embeddings: The M is mapped to a query matrix Q M , and their keys and values matrices K M and V M .