The truth data was created by asking NIST assessors (the humans performing this task) to identify the set of relevant sentences from each relevant document and then from that set of relevant sentences, mark those that were novel.
the assessors were instructed to identify a list of sentences that were: 1. relevant to the question or request made in the description section of the topic, 2. their relevance was independent of any surrounding sentences, 3. they provided new information that had not been found in any previously picked sentences.