Here we use GRU as the local feature extractor, and features produced for window x t h+1:t can be represented as: To maintain translation invariant, a max-overtime pooling layer is then applied to CNN or DRNN layer, the pooling result is regarded as the output of Encoder2: Set enc 1 as the global representation produced by Encoder1, required information for a certain window x t h+1:t with size h is defined as g t : where G is a function of interaction mode.
en-coder1 serves as a global information provider, while encoder2 performs as a local feature extractor and is directly fed into the classifier.