An important feature of our method is that while the model explicitly takes advantage of n-gram information, the model only outputs character-level encodings that is consistent with BERT.
downstream tasks are not affected.