(2018) interpreted lexical-semantic meanings and grammatical roles for each word by interpreting learned internal representation, which is in contrast to looking at the input patterns to interpret activated internal neurons.
interpreting a result by analyzing attention or neurons has a limitation that it cannot provide a rich explanation.