We used perplexity on development data to jointly choose 2 Our vocabulary and parameter set are determined from the lexicon.
(2) has 1 hidden layer, but we actually generalized this to consider networks with k ∈ {1, 2, 3, 4} hidden layers of d = 100 units each.