Gram–Schmidt process

The process goes one vector at a time. The first is kept, \(w_1=v_1\). Each later vector is corrected by removing its component along all the previous ones: \(w_2=v_2-\operatorname{proj}_{w_1}v_2\), \(w_3=v_3-\operatorname{proj}_{w_1}v_3-\operatorname{proj}_{w_2}v_3\), and so on. What remains is orthogonal to what was already built because each projection removes exactly the part that was not.

Guiding example: with \(v_1=(1,1,0)\) and \(v_2=(1,0,1)\), \(w_1=(1,1,0)\) and \(\frac{v_2\cdot w_1}{w_1\cdot w_1}=\frac12\), so \(w_2=(1,0,1)-\tfrac12(1,1,0)=(\tfrac12,-\tfrac12,1)\). One checks that \(w_1\cdot w_2=\tfrac12-\tfrac12=0\). To get unit vectors, divide each \(w_k\) by its norm, \(q_k=w_k/\|w_k\|\), which yields an orthonormal basis.

At every step, \(w_1,\dots,w_k\) span the same subspace as \(v_1,\dots,v_k\): only combinations of earlier vectors are subtracted. If some \(w_k\) comes out as \(\mathbf 0\), then \(v_k\) depended on the earlier ones, so the starting list was not independent. A typical mistake is projecting onto the original \(v_j\) instead of the already orthogonalized \(w_j\); separate projections add up correctly only when the reference vectors are orthogonal to each other.

This process shows that every finite-dimensional subspace has an orthonormal basis, and it underlies the factorization \(A=QR\), where \(Q\) stores the \(q_k\) and \(R\) the projection coefficients. It also simplifies least squares, since with an orthonormal basis the projection onto the subspace is a sum of projections onto each vector.