The historical spellings in the test set which are the inputs of the model are also case-sensitive.
they hypothesize that the reason why the combination of these two models does not gain more improvement is that the multi-task learning has already learned the attention patterns.