As mentioned in Section 1, we evaluate the two models on a measure of pragmatic informative-ness: how well does the model represent a human speaker, such that a generative model of a listener can be built from it to interpret utterances?
formally, for a speaker S(u | , c t , C) and an example consisting of an utterance, language identifier, and color context (u, , C), we identify the t * that maximizes the probability of u according to S: L uses a noisy-channel model with a uniform prior over target colors and S as a generation model to infer the most likely target color given the input utterance.