Then we obtain matrixÃ ∈ R U ×V : where K j ∈ R U ×d is a matrix whose rows are vector k j,c .Ã can be an arbitrary high rank because the righthand side of Equation 9 computes not only the matrix multiplication but also a nonlinear function.
an RNN language model with DOC can output a distribution matrix whose rank is identical to one of the true distributions.