It is common practice in modern NLP to represent text in a continuous space.
the interface between the discrete world of language and the continuous representation persists and is arguably nowhere so important as in deciding what should be the atomic symbols that will be used as input and/or output.