For each compared model, we chose the α value that showed the best self-normalization performance without sacrificing significant perplexity performance.
we used α DEV-LM = 1.0 and α NCE-R-LM = 10.0 for both PTB and WIKI datasets, and then α AND-LM = 1.0 and α AND-LM = 0.1 for the PTB and WIKI datasets, respectively.