Response candidates on the test set are collected by a standard search engine Apache Lucene 4 , other than negative sampling without human judgment on Ubuntu Dialogue Corpus.
the last turn of each Douban dialogue with additional keywords extracted from the context on the test set is used as query to retrieve 10 response candidates from the Lucene index set.