|d| means turns of a specific dialog d ∈ D. To deal with unsupported user actions, A s will be a subset of A s .
the KL term in equation (3) can be defined as follows: As the original policy parameters θ are fixed, the loss function in equation 3can be rewritten as: This objective will transfer knowledge of the original system to the "student" at the turn level.