Thus, we must be finding ∇Z by formulating it as a certain expectation r. Specifically, to be rs T , a matrix.
when using this semiring to compute second derivatives (Case 2) or covariances, one may exploit the invariant that r = s, e.g., to avoid storing s and to compute r1s2 + s1r2 in multiplication simply as 2 • r1r2.