As training proceeds, τ gets close to 0, yielding the increasingly peaked distribution that finally emulate the discrete cases.
the i-th word generated by the decoder is where W v ∈ R |V |×e denotes word embedding.