• The DDQ(K) agents are learned by DDQ of Algorithm 1, with an initial world model pretrained on human conversational data, as described in Section 3.1.
except for very simple tasks (Singh et al., 2002;Gašić et al., 2010Gašić et al., , 2011Pietquin et al., 2011;Li et al., 2016a;Su et al., 2016b), RL is too expensive to be applied to real users to train dialogue agents from scratch.