2 To gain further insights into our results, we examine the attention weights computed by FANs during verb-number prediction (supervised objective).
for each attention head at each layer of the FAN, we compute the percentage of times the subject is the most attended word among all words in the history.