The model processes text incrementally while learning which past tokens in the memory and to what extent they relate to the current token being processed.
the model induces undirected relations among tokens as an intermediate step of learning representations.