To compare with the video games, items were annotated using two additional methods: crowdsourcing and a non-video game with a purpose.
for both games, players were equally likely to select novel items, suggesting the games can serve a useful purpose of adding these missing relations in automatically constructed knowledge bases.