We find that encoder hidden states outperform word embeddings significantly which indicates that encoders adequately encode relevant information for disambiguation into hidden states.
to encoders, the effect of decoder is different in models with different architectures.