In this work, we cast information aggregation in multi-head attention as the problem of assigningparts-to-wholes.
each attention head extracts different linguistic properties of the same input (Raganato and Tiedemann, 2018), and the goal of information aggregation is to compose the partial representations extracted by different heads to a whole representation.