On Robust04, the use of smaller BERT variants always leads to decreasing effectiveness.
when using BERT-Small and BERT-Base for the final re-ranking, the corresponding BERT-QE variants always outperform BERT-Large significantly at the 0.1 level.