Note that in the 0-column, the 0 only refers to the ground-truth object not having a referring expression -the other objects may have expressions associated with them.
the supervised models can better determine when two expressions are incompatible, leading to the model choosing an object without any previous referring expression.