To simplistically visualize how heads cooperate in modeling localness, we propose an additional parametric model which is assigned a learnable but unified window size for each head, namely head-specific.
the window size D m of the m-th head is calculated as: where the scalar z m is a trainable parameter, N = 50 denotes a pre-defined constant number.