We compare the GA Reader as described here to a model which is exactly the same in all aspects, except that it passes document embeddings D (k) in each layer directly to the inputs of the next layer without using the GA module.
x (k) = D (k) for all k > 0.