NLP pipeline for stopword removal and stemming, resulting text is processed into unigrams) but instead of one-hot vectors, GloVe vector embeddings are used.
the main advantage of transformer-XL  is that it allows the capture of longer-term dependencies and resolves context fragmentation problem.