We created the following two Amazon Mechanical Turk tasks, which enable humans to play the language game on the scenarios: Speaker Task In this task, human annotators play the role of speakers in the language game.
utility (Communicative Success) We primarily evaluate a speaker by its ability to communicate successfully with a human listener.