We used perplexity on development data to jointly choose 2 Our vocabulary and parameter set are determined from the lexicon.
we create a regularized parameter ω , yielding a smoothed estimate p( ), even if the training count c( ) = 0. the learning rate, the initial random θ (from among several random restarts), the regularization coefficient λ ∈ {10 −1 , 10 −2 , 10 −3 , 10 −4 } and the neural network architecture.