Pre-training can bring about a significant improvement in performance on both in-domain and out-of-domain test sets; however, the in-domain improvement is significantly greater than that of out-of-domain when the two domains are far apart.
the difference between in-domain and out-of-domain in German and English is large, while the two domains in Czech are similar.