Then we have: The decoder's next state is updated by h i+1 = RNN(h i , w i+1 ), h i ) and h 0 is computed via summation-fusion.
attention fusion lets the decoder focus on different latent dimensions at each generation step.