The source-side context vector is fed to the decoder, so that translation at each step is conditioned on the whole source-side context.
we propose two types of sentential context: 1) the shallow one that only exploits the top encoder layer, and 2) the deep one that aggregates the sentence representations of all the encoder layers.