A bidirectional LSTM creates a representation g x of an input sequence x = (x 1 , x 2 , ..., x T ) by computing a forward sequence − → g and a reverse sequence ← − g over the input sequence, concatenating the results of the two sequences, and applying a rectified linear unit (ReLU) activation where f (x, y) is the output of an LSTM unit with inputs x and y.
we create a representation of the stem by taking the characters of (stem i,1 , stem i,2 , ..., stem i,K ) as the input sequence for a "stem" LSTM and a representation of the tags by taking (tag i,1 , tag i,2 , ..., tag i,L ) as the input sequence for a separate "tag" LSTM.