Hence, the written forms of the two languages should be very similar, which makes the language embeddings based on language modelling highly similar to one another.
when the embeddings are fine-tuned on a task taking orthography as well as phonology into account, this is no longer the case.