To evaluate the success of S 1 as compared to S 0 , we define a listener L eval (image|caption) / P S 0 (caption|image), where P S 0 (caption|image) is the total probability of S 0 incrementally generating caption given image.
l eval uses Bayes' rule to obtain from S 0 the posterior probability of each image w given a full caption u.