The reconstructor R reconstructs the original text x from y s .
we seek an output of reconstructor x that is as close to the original text x as possible; hence the loss for training the reconstructor, R loss , is defined as where the reconstruction loss l s (x,x) is the crossentropy loss computed between the reconstructor output sequencex and the source text x, or the negative conditional log-likelihood of source text x given word sequence y s sampled from G(x).