To encourage Q-BOT to ask diverse questions, we propose a simple auxiliary loss that penalizes similar dialog state embeddings.
given Q-BOT states -s Q t´1 , s Q t -in addition to maximizing likelihood of question (during supervised pretraining), or image-guessing reward (during selftalk RL finetuning), we maximize a smooth-L1 penalty on resulting in ř N t"2 f p∆ t q as an additional term in the overall objective (N " no.