This state encodes the word given its sentential context.
the first layer, BiGRU , maps the sequence of input vectors (x 1 , ..., x T ), into a sequence of states (s 1 , ..., s T ), the second, BiGRU , transforms those states into (s 1 , ..., s T ), and the third, BiGRU , produces the final representations of the words in context (s 1 , ..., s T ).