For each token in the sequence, we perform a softmax over all candidate tags to output the most likely tag: Our network structure as shown in Figure 2 also contains connections from the output y t−1 of the previous time step to the current top hidden layer.
we train our network by maximizing the logprobability of the correct entity E and relation R tag sequences jointly given the sentence S as below: we can decompose our objective into the sum of log-probabilities over entity sequence and relation sequence.