Another possible explanation is that there is no clear relation between the context space and the target function: the context space is learned only indirectly by backpropagation.
due to the random initialization of the parameters and to data sparsity, many vectors of R might be blocked in some local maxima, meaning that similar vectors cannot be grouped in a consistent way and that the induced similarity is more "loose".