The idea is to have a set of language expert networks, one per source language, each responsible for learning language-specific features for that source language during training.
instead of hard-switching between the experts, each sample uses a convex combination of all experts, dictated by an expert gate.