Comparing encoder representations to decoder representations, it is interesting to see that in several cases the decoder side representations performed better than the encoder side ones, even though the former were trained using a uni-directional LSTM.
since there is no difference in the general trends between the encoder-and the decoder-side representations, below we focus on the encoder-side only.