In comparative terms (and focusing on perplexity and rank, given the uniformly low accuracy results) we observe a stronger performance of the traditional N-Gram models over the neuralnetwork-based ones, possibly pointing to the difficulty of tuning the latter properly.
the best relative performance on LAMBADA is achieved by N-Gram w/cache, which takes passage statistics into account.