From this point, we can leverage the pair-wise agreement of two output layers to add pseudo-labeled examples as training data to the third model.
mT-Tri only necessitates one forward pass as well as the evaluation of the two additional output layers (which takes a negligible amount of time) and requires about as many epochs as tri-training until convergence (see Table 3, second column) while adding fewer unlabeled examples per epoch (see Section 3.4).