, E ln } ∪ {E en }, which are pre-trained on large corpora using any standard word embedding toolkit.
e en represents the pre-trained embedding for english, and e l i represents the pre-trained embedding for non-english language l i at the subword level.