To overcome the above-mentioned disfluencies in existing word salience scores, we propose an unsupervised method that first randomly initialises word salience scores, and subsequently updates them such that we can accurately predict the words in local contexts.
we train a two-layer neural network where in the first layer we take pre-trained word embeddings of the words in a sentence S i as the input and compute a representation for S i (here onwards referred to as a sentence embedding) as the weighted average of the input word embeddings.