That is, giving both s i,i+1 and s i,i−1 large values, so the attention head freely attends to any position without restriction of constituent prior, which is the same as the original Transformer.
the tree structures can be automatically induced from raw texts by our proposed "Constituent Attention" module, which is simply implemented by self-attention between two adjacent words.