The major difference is that they directly use absolute positions, whereas ours utilizes relative positions.
their model needs to re-encode the partial sequence at every step, which is computationally more expensive.