The switching network, which is a multilayer perceptron in our experiments, takes the representation of the context sequence (similar to the input annotation in NMT) and the previous hidden state of the output RNN as its input.
at each time-step t, the softalignment mechanism first computes the relevance weight e tj which determines the contribution of annotation vector h j to the t-th target word.