This behavior of gracefully falling back to NCE is more desirable than the alternative of stalled training if p − (y|x) does not have a simple p nce mixture component.
we would still like to avoid such collapse, as the adversarial samples provide greater learning signals than NCE samples.