Optimality and least squares

Concepts

• Orthogonal projection from Linear algebra 10 exercisesThe orthogonal projection of a vector onto a nonzero vector is the multiple of the latter closest to the former; the remainder is…
• Function of several variables from Multivariable calculus 10 exercisesA function of several variables f assigns to each point mathbf x=(x_1,dots,x_n) of a set D⊆mathbb R^n, its domain, a single real…
• Gradient from Multivariable calculus 10 exercisesThe gradient ∇ f of a differentiable function f is the vector that collects its partial derivatives at a point; it points where f…
• Critical point of a multivariable function from Multivariable calculus 10 exercisesA critical point of a function f of two variables is a point (a,b) of its domain where the gradient ∇ f is the zero vector or doe…
• Optimization problem 10 exercisesAn optimization problem consists of minimizing or maximizing an objective function f over a set of points that satisfy certain co…
• Local and global minima of convex functions 10 exercisesA local minimum of a function is a point where no sufficiently small neighborhood contains smaller values, while a global minimum…
• First-order necessary condition 10 exercisesA first-order necessary condition states that if x^* is a local minimizer of a differentiable function f, then ∇ f(x^*)=0.
• Linear least squares 10 exercisesOptimization problem that minimizes the sum of squared residuals \|Ax-b\|_2^2 to approximate a vector b by linear combinations of…