We observe 4 Statistical significance of p < 0.01 for CIDEr-D and ROUGE-L, p < 0.02 for BLEU-4, p < 0.03 for METEOR, based on the bootstrap test (Noreen, 1989;Efron and Tibshirani, 1994) with 100K samples.
directly generating the entailed hypothesis sentences given a premise sentence would be even more beneficial than retrieving or reranking sentence pairs, because most downstream generation tasks only come with the source sentence and not pairs.