We approximate this marginalization by constructing a tree where the root is the predicate, p, the branches are likely sequences of arguments, and the leaves are the word and label for which we need to estimate a probability, w:l. Formally, we define this tree of possible sequences as: where w f,0 :l f :0 = p:P RED; k and T are thresholds; and argmax k (q) is the k word:label pairs that have the highest probability of being the next argument given the sequence q according to the PRNSFM.
as shown in Figure 2, we use two different embedding layers, one for word values and one for semantic labels, and the two embedding vectors are concatenated before being passed to the LSTM layer.