Using t and h(s i , s j ) above, we compute the cross-entropy error E(t, (S i , S j )) for an instance (t, (S i , S j )) as follows: Next, we backpropagate the error gradients via the network to compute the updates as follows: Here, we drop the arguments of the error and simply write it as E to simplify the notation.
in principle, we can learn NWS scores using any pre-trained set of word embeddings.