If our neural network has only one hidden layer and is self-normalized, the only remaining computation is 512 calls to tanh() and a single 513-dimensional dot product for the final output score.
6 only ∼3500 arithmetic operations are required per n-gram lookup, compared to ∼2.8M for self-normalized NNJM without precomputation, and ∼35M for the standard NNJM.