Character-level NMT systems have a more difficult sequence-modeling task, as they need to infer the meaning of words from their constituent characters, where models with larger tokens instead delegate this task to the embedding table.
we hypothesize that increasing the model's capacity by adding layers will have a greater impact on character-level models.