Our intuition is that modeling interactions through the output spaces rather than hoping that the encoder somehow learns to capture them, provides a useful inductive bias to the model.
capturing this interaction explicitly should mitigate overfitting.