By contrast, in soft attention mechanism, a probability distribution which reflects the importance of tokens is calculated over each token of the input sequence.
furthermore, we introduce a novel trick to reduce the high variance in training brought by REINfORCE method.