The feature that distinguishes our model most from previous work is that we apply a bidirectional recurrent layer (LSTM) on all characters of a sentence to induce fully context sensitive initial word encodings.
we do not restrict the context of this layer to the words themselves (as in Figure  1b).