This is true for the training of the model as well as for applying it when computing the representation of a new text.
to prior work that has assumed that the sequence-of-character information captured by character ngrams is sufficient, position embeddings also capture sequence-of-ngram information.