The attention weight score is a global score over the words and the vector dimensions; whereas in our model, there are N word × N filter × N dimension gate outputs.
we train a small model with only one filter which is only three word wide.