Slavic BERT The Slavic model 4 was built using transfer learning from the Multilingual BERT model to four Slavic languages: Bulgarian, Czech, Polish, and Russian.
the Multilingual BERT model was fine-tuned on a stratified dataset of Russian news and Wikipedia articles for the other languages.