Definition 1 (Expected Zero-shot Consistency) Let E s and E 0 be supervised and zero-shot tasks, respectively.
let (•) be a non-negative loss function and M be a model with maximum expected supervised loss bounded by some ε > 0 : we say that a machine translation system is zero-shot consistent if low error on supervised tasks implies a low error on zero-shot tasks in expectation (i.e., the system generalizes).