With enough GRE data, the prediction error may be backpropagated to directly adjust or learn embedding in the look-up tables.
given the limited size of the GRE data, we only employed the top hidden layers to non-linearly merge the distances between a word pair that are obtained within each of the modules in the Contrast Inference Layer.