W i denotes the i-th column of the weight matrix W .
when we want to calculate the contribution of a subset of characters, where S is the set of corresponding character position indexes and S ⊆ {1, ..., T }, we should decompose the output of the filter z t into three parts: the relevant contribution β t originating from the selected subset of characters with indexes S, the irrelevant contribution γ t originating from the remaining characters in the sequence, and a bias which is deemed neutral (Murdoch et al., 2018).