The disentanglement system's test performance decreases proportionally; mean 1-to-1 falls to 36.08, and mean loc 3 to 63.00, essentially baseline performance.
if we run our classifier on a pair of very distant utterances, we expect it to default to the majority class, which in this case will be different, and this will damage our performance in case the two are really part of the same conversation.