For each time step, our reinforcement learning agent predicts the zero pronoun-candidate antecedent pair by using 1) the zero pronoun; 2) information of current candidate antecedent and 3) antecedent information generated by antecedents predicted in previous states.
our reinforcement learning agent is designed as a policy network ⇡ ✓ (s, a) = p(a|s; ✓), where s represents the state; a indicates the action and ✓ represents the parameters of the model.