Formally, g(x) can be expanded as: If we treat the bias term as b × x 0 where x 0 = 1, then the weight corresponding to each feature combination x i x j x k is w i w j w k , which is exactly the same as a rank-1 component tensor in the lowrank form using CP tensor decomposition (Cao and Khudanpur, 2014).
the cube activation function implicitly derives full feature combinations.