However, the characters in these tokens are probably very informative (for example "http" in a token would signal that the token is a URL).
fastText, which uses n-grams from a token to compute embeddings, would emit more meaningful representations.