Motivated by the nearest neighbor aware semi-supervised framework [Shi et al., 2018], we artificially generate pseudo-labels for respective unpaired datasets.
we retrieve a best match captioñ y i in D y u given a query image x i , and assign it as a pseudo-label, and vice versa (x i for y i ).