Firstly, teacher-student learning can probably help to learn label preferences for some specific words in the target language.
if a word appears in the unlabeled target-language data and the teacher model consistently predicts it to be associated with an identical label with high probabilities, the student model would learn the preferred label w.r.t that word, and predict it in cases where the sentence context may not provide enough information.