While these models differ in how the context vector c t is derived, they share the same subsequent steps.
given the target hidden state h t and the source-side context vector c t , we employ a simple concatenation layer to combine the information from both vectors to produce an attentional hidden state as follows: The attentional vectorh t is then fed through the softmax layer to produce the predictive distribution formulated as: We now detail how each model type computes the source-side context vector c t .