The rationale behind fixing the set V is a) to make the language model more robust to handling OOVs and to effectively bypass the problem of unreliable word estimates for low-frequency and unseen words (by ignoring them), and b) to enable direct comparisons of absolute perplexity scores across different models.
this posits a critical challenge as cross-linguistic evaluation becomes uneven.