We found it useful to have two hyperparameters per auxiliary task governing the number of distinct named entity word types and tokens used in defining the losses L 1 and L 2 .
per document these hyperparameters control in a top-tobottom order the number of distinct named entity word types we attempt to predict, as well as the number of tokens of each type considered.