We studied the impact of using different representation units-words, characters, BPE units, and morphological segments-on the representations learned by seq2seq models trained for neural machine translation.
we evaluated the performance of such representations on core natural language processing tasks modeling morphology, syntax, and semantics.