In consequence, a single-layer disentangled self-attention with a large number of heads is difficult to handle the complexity.
we further propose to stack multiple DSA layers to better model the huge semantic space with strong compositionality.