We can then write unary Thus we can regard the bilinear form as a function computing a weighted inner product over some real embedding v y U representing state y and some real embedding representing input factor t. The rank of W gives us the intrinsic dimensionality of the embedding.
if we want to induce shared low-dimensional embeddings across different states it seems reasonable to impose a low rank penalty on W .