The convolution layer applies a filter w j ∈ R (h+1)k to each ngram window of h+1 tokens.
writing x i:i+h for the concatenation of the word vectors x i , x i+1 , ..., x i+h , we have: where b j ∈ R is a bias term and f is the ReLu activation function (Nair and Hinton, 2010).