Through well-designed probing experiments, we empirically validate that the improvement of SSANs can be attributed in part to mitigating two commonly-cited weaknesses of SANs: word order encoding and structure modeling.
the selective mechanism improves SANs by paying more attention to content words that contribute to the meaning of the sentence.