Figure 2 shows that Transformer with hard attention is more effective in handling the long sequences.
the performance gap between our model (THA) and the original Transformer model (TSA) grows bigger as sequences become longer.