In Proceedings of the 20th ACM international conference on Information and knowledge management, CIKM.
to generate the golden standard, given a query, the judges first manually generate a candidate list of external events 7 ; then for each event, they look into the tweets within the corresponding period and check whether there is a surge on the frequency of tweets.