Besides, for T P and T N , we approximate them using their expectation on the current batch, which can produce a robust estimation even when the batch size is not large enough.
let i=1 } is N B negative instances in the batch, we estimate T P (θ) and T N (θ) as: T N B (θ) = Then we can compute the estimatorŵ β (θ) for w β (θ) as:ŵ whereŵ β (θ) is computed using only the instances in a batch, which makes it can be directly applied as a plug-in of conventional batch-based neural network optimization algorithm where the loss of negative instances in batch are scaled byŵ β (θ).