The major improvements come from contextualized BERT embeddings rather than word2vec and ELMo, and from fine-tuning BERT on large NLI datasets.
we also observed that soft alignments (MoverScore) marginally outperforms hard alignments (BERTScore).