Fine-tuned ELMo has an obvious improvement in Sen-Making and a non-obvious improvement in Explanation, probably because introducing knowledge will help models to identify common sense but cannot help them in inference.
fine-tuning makes BERT perform the same in Sen-Making and even worse in Explanation.