The latter is defined as a concatenation of the hidden states for each input vector obtained from the forward GRU, g w , and the backward GRU, g w : The same concatenation is applied for the hiddenstate representation of a sentence h A typical way to obtain a representation for a given word sequence at each level is by taking the last hidden-state vector that is output by the encoder.
it is hard to encode all the relevant input information needed in a fixed-length vector.