probability for e, we define: where w t ∈ R h is the output weight vector and b t the bias scalar for type t, and a t is the aggregated representation of all contexts c i of e for type t, computed as follows: where α i,t is the attention score of context c i for type t and a t ∈ R h can be interpreted as the representation of entity e for type t. α i,t is defined as: where M ∈ R h×dt is a weight matrix that measures the similarity of c and t. t ∈ R dt is the representation of type t. Table 1 summarizes the differences of our MIML methods with respect to the aggregation function they use to get corpus-level probabilities.
for optimization of all MIML methods, we use the binary cross entropy loss function, to the loss function of distant supervision in Eq.