Note that this formulation is in similar spirit to highway networks (Srivastava et al., 2015).
since our gating function is learned via reasoning over multi-granular sequence blocks, it captures more compositionality and long range context.