The teacher-student optimization only adjusts the weights related to the language-specific parts in Figure 3 (i.e., utterance encoding and context gating).
the shared weight ||W y || is seen as a constant.