Input Embedding: To apply deep-learning method to text data, we first map each word into a lowdimensional continuous vector, which is also known as word embedding (Mikolov et al., 2013;Pennington et al., 2014).
let L ∈ R d×|V | be an embedding matrix made up of all the word embeddings, where d is the dimensionality of word embedding and |V | is vocabulary size.