For example, the context word set for As both hidden states and contextual words are represented as real-valued vectors, we need to factorize vector-level relevance at the neuron level.
the neuron-level relevance between the m-th neuron in a hidden state v m ∈ R and the n-th neuron in a contextual word vector u n ∈ R is denoted as r un←vm ∈ R, which satisfies the following constraint: the vector-level relevance between a hidden state v and one contextual word vector u ∈ C(v) is denoted as R u←v ∈ R, which quantifies the contribution of u to the generation of v. It is calculated as Definition 4 the relevance vector of a hidden state v is a sequence of vector-level relevance of its contextual words: our goal is to compute relevance vectors for hidden states in a neural network, as shown in Figure 2.