As k is usually small (averaging around 2 per example in our experiments), sequence lengths remain similar to those encountered for the same x during language modeling.
using LMs to directly predict x from x˜ as in Fedus et al. (2018) effectively doubles the sequence length of x