All of the parameters of our model including word embeddings were uniformly initialised in [-0.1, 0.1], and gradient clipping (Pascanu et al., 2013) was used with the clipping value 5.0.
the last condition is hard to assume when dealing with resource-poor languages, for which unsupervised methods can be beneficial in reality.