Compared with diverse decoding (Li et al., 2016c), we observed during development that sentences generated by different dropouts tend to have diverse semantics (hence more likely to have different plausibility levels).
sentences from diversity decoding often have similar structure and phrases across candidates.