The translation model computes a distribution over the V possible target words y i+1 by transforming the LSTM output h i via a linear layer with weights W o and bias b o : The conditional input c i at time i is computed via a simple dot-product style attention mechanism (Luong et al., 2015a).
we transform the decoder hidden state h i by a linear layer with weights W d and b d to match the size of the embedding of the previous target word g i and then sum the two representations to yield d i .