This is because the same per-language parameters are used to translate to and from a given language, irrespective of the language at the other end.
as long as we train our model using some language pairs that involve a given language, it is possible to learn to translate in any direction involving that language.