To train the neural network, we optimize the following objective: where N is the number of training examples, trg i is the target vector representation for the ith example learned by RAE and p i is the output of the neural network for the source vector representation src i of ith example.
we randomly selected 5M examples as training set, 10k examples as development set and 10k examples as test set.