The attention mechanism firstly collects the context information by multiplying trainable weights with all the vectors from every time step of the last LSTM output.
by only using WE + POS as the features, combined with DBLSTM, the model can achieve compelling result.