Finally, the internal memory state, m t and new output state h t , of the LSTM at t are computed as where indicates elementwise vector multiplication and φ is, as before, some non-linear function (we use tanh).
g i determines to what extent the new input word is considered at each time step, g f determines to what extent the existing state of the internal memory is retained or forgotten in computing the new internal memory, and g o determines how much this memory is considered when computing the output state at t. The sentence-final memory state of the LSTM, m N , a 'summary' of all the information in the sentence, is then projected via an extra non-linear projection (parameterised by a further weight matrix) to a target embedding space.