Although most held-out irregulars are regularized, as expected, the ED model does, perhaps surprisingly, correctly conjugate a handful of irregular forms it has not seen during training-five in the test set.
three of these are prefixed versions of irregulars that exist in the training set (retell → retold, partake → partook, withdraw  Figure 1: Single-task vs. multi-task.