Since BERT relies heavily on the learned attention weights, we define disabling a head as modifying the attention values of a head to be constant a = 1 L for every token in the input sentence, where L is the length of the sentence.
every token receives the same attention, effectively disabling the learned attention patterns while maintaining the information flow of the original model.