The best permutation between hypotheses and reference labels in terms of cross-entropy is selected and used for backpropagation.
this method still requires reference labels in the form of senone alignments, which have to be obtained on the clean isolated sources using a single-speaker ASR system.