This work targets these shortcomings by designing a model and training scheme targeted towards entity tracking.
we introduce Figure 1: A LAMBADA example where the final word "julie" (with reference chain in brackets) is the answer, y, to be predicted from the preceding context x.