We address this by reranking the positively labeled sentences with the probability scores obtained from the softmax layer (rather than the label itself).
we are more interested in is the relative ranking of each sentence rather than their exact scores.