Distinct-1 and distinct-2 are widely used in the literature (Li et al., 2016a;Shen et al., 2018a;Xu et al., 2018b), measuring the ratio of unique unigrams/bigrams to the total number of unigrams/bigrams in a set of responses.
they are very sensitive to the test data size, since increasing the number of examples in itself lowers their value.