To prevent overfitting, we tuned the parameters using cross-validation on 80% of PL05 dataset (α = 5, λ 1 = 1 for PK since there is no need for distortion as the phrases are of the same length by definition, and λ 1 = λ 2 = 0.5 for CK) and used the same set of parameters on the remaining datasets.
we believe CK has more room for improvement with the use of more accurate phrase embeddings such as the ones from Le and Mikolov (2014), Yin and Schütze (2014) and Yu and Dredze (2015).