The proposed pre-training scheme involves learning the parameters associated with h a and h p using two natural subtasks: anaphoricity detection and antecedent ranking.
we (1) train h a on the task of predicting whether a particular mention is anaphoric or not, and (2) train h p on the task of predicting the antecedent of mentions known to be anaphoric.