The PRPN computes the difference between the distance d t at the current step and all previous steps d j for 2 ≤ j < t. The difference is normalized to the range [0, 1]: where τ is the temperature.
if   Predict next word Gated-weighted attention Figure 3: The prediction of the next word in the PRPN language model.