Figure 3 shows that we extend the basic ROP architecture ( §4) by a GRU layer (bottom layer in Figure 3), denoted as GRU r , that learns the representation of the relation sequence.
finally, we use h t (the last hidden state of GRU r , shown in blue in figure 3) to match the query relation r. 3 the training loss of this task has two parts: one comes from the loss of ROP training (l seq , Eq 18); the other is the prediction loss of query relation, denoted as l pred .