4 becomes: 4 Implementation and Dataset We modified the Feature-Augmented RNN toolkit 2 and adapted it to handle tree-structured data.
and instead of being run sequentially on the entire training corpus, the RNN is run on all the word tokens in all unrolls of all the sentences in all the books of the corpus.