The two tasks took the form of "intruder" tasks, whereby subjects were tasked with identifying an intruder topic word for a given topic, or an intruder topic for a given document.
in the word intrusion task, an intruder word was added to the top-5 topic words, and annotators were asked to identify the intruder word.