For the base neural network we use the "nonstatic" version in (Kim, 2014) with the exact same configurations.
word vectors are initialized using word2vec (Mikolov et al., 2013) and fine-tuned throughout training, and the neural parameters are trained using SGD with the Adadelta update rule (Zeiler, 2012).