In this work, we focus on whether the attention weights' suggested importance ranking of I faithfully describes why the model produced its output, echoing existing work on explanation brittleness for other model components (Ghorbani et al., 2017).
one point worth noting is the facet of interpretability that our tests are designed to capture.