We observed that the naive pairwise ranking loss that uses a fixed margin unfairly penalizes the locally positive sentences during training.
the main differences between our implementation and the implementation referred in their paper are that we used a bi-LStM (as opposed to simple RNN) for sentence encoding and trained the network with the Adam optimizer (as opposed to AdaGrad).