Thus, for simplicity, assume that we have a single row and want to minimize The minimum is either at w = 0 (the tip of the cone) or where the partial derivatives are zero (Figure 3): Clearly, w and v must have the same direction and differ only in magnitude, that is, w = α v v .
substituting this into the above equation, we get the solution the update is As above, since the ∞,1 norm on matrices (2) is separable into the ∞ norm of each row, we can treat each row separately; thus, we want to minimize Figure 3: Examples of the two possible cases for the 2 gradient update.