As the pun data set is limited, the pun language model has no creativity, which means if we input a word appearing in the training data, then the output will usually be an existing sentence from the training data.
we remove the sentences which contain words in the 100 target words from the pun data set, and then train the model for test.