For a target word w i , the objective function of MGE is almost equivalent to maximizing P (w During the backpropagation, the gradients of h i 1 , h i 2 , h i 3 can be different in our model while they are always same in MGE, so the gradients of the embeddings of words, characters, subcharacter components can be different in our model while they are same in MGE.
the representations of words, characters, and subcharacter components are decoupled and can be better trained in our model.