Newton is spectacular near a solution, but full steps can fail badly away from it. Compare its local behavior with examples where globalization is essential.
Optimization landscapeclick the contour plot to choose a new initial point
ConvergenceEuclidean distance to the nearest global minimizer
Discrete iterations
20
Quadratic convergence really is that fast.
Near a global minimizer, the Hessian is positive definite and unit-step Newton enters its classic local regime: the error is essentially squared at every step. In contrast, fixed-step GD still crawls along at a linear rate.