Least Squares Data Fitting

1. What does the least-squares solution minimize?
2. For a given model and given data points, can you form the system \({\bf A} {\bf x} \cong {\bf b}\) for a least squares problem?
3. For a small problem, given some data points and a model, can you determine the least squares solution?
4. In general, what can we say about the value of the residual for the least squares solution?
5. What are the differences between least squares data fitting and interpolation?
6. Given the SVD of a matrix \({\bf A}\), how can we use the SVD to compute the residual of the least squares solution?
7. Given the SVD of a matrix \({\bf A}\), how can we use the SVD to compute the least squares solution? Be able to do this for a small problem.
8. Given an already computed SVD of a matrix \({\bf A}\), what is the cost of using the SVD to solve a least squares problem \(A {\bf x} \cong {\bf b}\)?
9. Why would you use the SVD instead of normal equations to find the solution to \({\bf A} {\bf x} \cong {\bf b}\)?
10. Which costs less: solving a least squares problem via the normal equations or solving a least squares problem using the SVD?
11. What is the difference between a linear and a nonlinear least squares problem? What sort of model makes it a nonlinear problem? For data points \((t_i, y_i)\), is fitting \(y = a \cos(t) + b\) where \(a\) and \(b\) are the coefficients we are trying to determine a linear or nonlinear least squares problem?