As output, a 600-dimensional attention layer was included prior to the final output, allowing the model to learn what to focus on from a sequence overall, based on a weighted combination of all the input states to the layer.
α it , the particular 'importance' of a word vector, is calculated through the activation of the hidden layer for that word, u it , and a learned context vector, u w , input through a softmax function.