At each step, each h i captures an increasing larger ngram context, while additionally communicating globally to all other h j via g. The optimal number of recurrent steps is decided by the end-task performance, and does not necessarily scale with the sentence size.
s-LsTM can potentially be both more efficient and more accurate compared with BiLsTMs.