For each query, we invite two senior graduate students to manually identify their golden bursty intervals, and each bursty interval is represented as a pair of timestamps in terms of hours.
to generate the golden standard, given a query, the judges first manually generate a candidate list of external events 7 ; then for each event, they look into the tweets within the corresponding period and check whether there is a surge on the frequency of tweets.