We integrate the two and acquire the final normalized attention weight v * ∈ R 1×(T −1) by feeding their elementwise product into the softmax function.
the main prediction can benefit from temporally-close hypotheses have been made and we decode our main hypothesisỹ T as where W T is a weight matrix and b T is a bias.