Both decision makers have access to the image and implicitly to the linguistic dialogue history: DM1 exploits the dialogue encoding learned by QGen's LSTM, which is trained to record information relevant for generating a followup question.
dM2 leverages the dialogue encoding learned by the Guesser's LSTM, which is trained to capture the properties of the linguistic input that are relevant to make a guess.