They also used segment embeddings (along with word and position embeddings) to differentiate between a pair of sentences in different languages.
this is only used in one of the pre-training phases of the language model (translation language modelling) and not in the downstream task.