If we rewrite the two-layer architecture in a compact way, we obtain: As the equation suggests, if we minimize the negative log likelihood, the loss function is not convex.
we could land in a non-optimal local minimum using online learning.