Similarly, we clip the distance with a maximum threshold of k (which is empirically set to 10), but we do not discriminate positive and negative values.
instead, since we do not want the model to be aware of directional information, we use the absolute values of the position differences: the learnable relative postion representations have k+1 types rather than 2k+1: we have , and .