P (y|x; θ) is defined on a holistic neural network which mainly includes two core components: an encoder encodes a source sentence x into a sequence of hidden representations H x = H 1 , ..., H M , and a decoder generates the n-th target word based on the sequence of hidden representations: where s n is the n-th hidden state on target side.
the model parameters of NMT include the parameter sets of the encoder θ enc and the decoder The standard training objective is to minimize the negative log-likelihood of the training corpus Due to the vulnerability and instability of deep neural networks, NMT models usually suffer from a drawback: small perturbations in the input can dramatically deteriorate its translation results.