The term-document matrix is typically very sparse with z nm non-zero entries while k is typically also much smaller than n, m. By using sparse matrix multiplications and avoiding dense intermediate matrices, the updates can be very efficiently and easily implemented.
updating F, S, G each takes O(k 2 (m + n) + kz) time per iteration which scales linearly with the dimensions and density of the data matrix.