More formally, let us illustrate the model by taking the sequence prediction task (Figure 1) as illustration.
given an utterance with labels y 1 , .., y n , our Multi-task Tri-training loss consists of three task-specific (m 1 , m 2 , m 3 ) tagging loss functions (where h is the uppermost Bi-LSTM encoding): to classic tri-training, we can train the multi-task model with its three model-specific outputs jointly and without bootstrap sampling on the labeled source domain data until convergence, as the orthogonality constraint enforces different representations between models m 1 and m 2 .