In our Python implementation this takes several minutes, which in principle should be amortized over the cost for each sentence.
just as Zens and Ney (2007) do for phrase tables, we could compile our data structures into binary memory-mapped files, which can be read into memory in a matter of seconds.