This sparsity helps the model to scale well to larger vocabulary, but it cannot model relations in translation beyond word cooccurrences.
we first tried with a fixed learning rates, but the results were disappointing.