We use a constrained brute force ranking scheme to evaluate all possible groundings (with a restricted granularity) between an image and its caption.
we divide the image into a grid, and extract all of the image crops whose boundaries sit on the grid lines.