On the word level, for instance, it is obvious that the classifier needs to combine the embeddings of the source and target word to make an informed decision and not merely calculate a weighted sum of them.
we opt for an architecture with hidden layers instead (see Fig.