Compared to visual dialog (Das et al., 2017a,b;Strub et al., 2017;Mostafazadeh et al., 2017) tasks, agents need to additionally cooperate to change the environment with actions (e.g., move pieces around).
the agents have to possess the ability to adapt and hold a dialog about partially-constructed scenes that will occur over the course of their interactions.