Pre-trained word embeddings are often seen to have positive impact on sequence-to-sequence frameworks (Cho et al., 2014a;.
traditional embeddings are not good at capturing relations like synonymy (Tissier et al., 2017), which are essential for simplification.