In general, even when a simple linear function generates the training data, these methods are not guaranteed to be consistent with it (Rot99a).
if the feature space is chosen so that they are, the robustness properties shown above become significant.