The second models is BLSTM extended with an attention mechanism (Rocktäschel et al., 2016;Golub and He, 2016) combined with convolution layers over the input.
the input em-  bedding layer is convoluted using 4 different convolution sizes (2, 3, 5, 7), each with 1,000 randomly initialized weight vectors.