One of the desired properties of multi-head attention is its ability to jointly attend to and encode different information in the embedding space.
there is no mechanism to support that these attention heads indeed capture diverse information.