We use a hidden size of 3056 and set the initial factorization dimension r of the parameter matrices to 512.
we replace each weight matrix W in SRU using an explicit factorization PQ with an inner dimension of 512.