Both the top-level and low-level policies can be learned with deep Q-learning methods, like DQN.
the top-level dialogue policy estimates the optimal Q-function that satisfies the following: where N is the number of steps that the low-level dialogue policy (intra-option policy) needs to accomplish the subtask.