An alternative to deterministic text generation is to sample directly from the softmax distribution.
since the probability mass tends to accumulate in a long tail, this procedure generates unlikely words too often, leading to degenerate text (Fan et al., 2018;Holtzman et al., 2020).