In the first stage, a text representation model ViBERT is trained in the bi-modal sequence-tosequence approach for scene layout generation on COCO.
visual commonsense knowledge like spatial relations will be encoded in ViBERT by the supervision of caption and image layout.