Finally, we concatenate all feature vectors of different h's for each i as the new word vector: The second part of HCWE is based on an RNN model which is used when texts are longer and include word dependencies that may not be captured by the CNN model.
we use a bidirectional LSTM and concatenate the forward and backward hidden state vectors as the new word vector, as shown below: The answer to the question whether to use local or global context to encode words for sentiment classification is still unclear, and both CNN and RNN models have previous empirical evidence that they perform better than the other (Kim, 2014;McCann et al., 2017).