However, this is very difficult given the large number of potential candidates, especially those with slight variations.
following the Concept Definition Retrieval task in (Guo and Diab, 2012b) and (Steck, 2010) we use a metric for evaluating the ranking of the correct news article to evaluate the systems, namely, ATOP t , area under the TOPK t (k) recall curve for a tweet t. Basically, it is the normalized ranking ∈ [0, 1] of the correct news article among all candidate news articles: ATOP t = 1 means the url-referred news article has the highest similarity value with the tweet (a correct NARU); ATOP t = 0.95 means the similarity value with correct news article is larger than 95% of the candidates, i.e.