4For entity i, we only compute attention weights to all its direct-connected neighbors, and normalize them with SoftMax.
so the normalized attention weight is Now we can compute the graph attention part for the state, which is simply the weighted sum of all neighbors' embedding on the attention space with the state vector s i,t of entity i in time t is which in turn inputs a three-layer feedforward neural network, whose final output is a softmax probability with length equal to the number of all relations in KG.