It seems that syllable-aware language models fail to outperform competitive character-aware ones.
usage of syllabification can reduce the total number of parameters and increase the training speed, albeit at the expense of languagedependent preprocessing.