Clearly, the image features and the corresponding similarity measures used in retrieval play a significant role here but, as investigated in , what makes this particularly difficult is that while describing an image humans do not explicitly mention every detail.
some parts of an image are more salient than the others.