The sentence encoder takes pre-trained fixed word embeddings, randomly initialized lemma embeddings, part-ofspeech and named-entity tag embeddings.
their model is very data demanding as they needed to train it on additional 100K sentences parsed by other parsers.