For fair comparison, we evaluated the best performed (on dev data) model during 500K training batches on the test data.
the updated objective function using weighted sampling J ws can be formulated as follows: only 80% of the entire corpus is used to perform the NMt training during each iteration (for the first two iteration, all of the sentences should be sampled).