Orthogonality · Definition

Orthogonal projection

linear algebraorthogonality

Before you start you need to know

It solves the problem of splitting a vector into a part along a given direction and a part perpendicular to it.

Definition

The orthogonal projection of a vector onto a nonzero vector is the multiple of the latter closest to the former; the remainder is perpendicular to the latter.

\[\operatorname{proj}_{v}u=\frac{u\cdot v}{v\cdot v}\,v\]

The idea is to find the multiple \(c\,v\) closest to \(u\). We require the remainder \(u-cv\) to be orthogonal to \(v\), that is, \((u-cv)\cdot v=0\), and solving gives \(c=\frac{u\cdot v}{v\cdot v}\). Thus \(u=\operatorname{proj}_v u+(u-\operatorname{proj}_v u)\) is the decomposition of \(u\) into a part parallel and a part perpendicular to \(v\).

Worked example: for \(u=(4,2)\) and \(v=(1,1)\), \(u\cdot v=6\) and \(v\cdot v=2\); the projection is \(3(1,1)=(3,3)\) and the remainder is \((1,-1)\), which satisfies \((1,-1)\cdot(1,1)=0\). The length of the projection is \(|u\cdot v|/\|v\|\), and the length of the remainder is the distance from \(u\) to the line spanned by \(v\).

The projection depends only on the line spanned by \(v\): using \(v\) or \(5v\) gives the same result. It is also linear in \(u\), it is zero exactly when \(u\perp v\), and it fixes the vectors on the line (projecting twice is projecting once). A typical mistake is forgetting to divide by \(v\cdot v\), which is only harmless when \(v\) is a unit vector.

This construction is the basic block of the Gram–Schmidt process, which applies it again and again to produce orthogonal bases, and of least squares, where one projects onto larger subspaces.

Think about it

If \(\|v\|=1\), how does the projection formula simplify?

Since \(v\cdot v=1\), it becomes \(\operatorname{proj}_v u=(u\cdot v)\,v\).

Why is projecting twice onto \(v\) the same as projecting once?

Because the first projection is already a multiple of \(v\), and every multiple of \(v\) is its own projection.

Exercises 10

  1. Compute the orthogonal projection of \(u=(3,4)\) onto \(v=(1,0)\).
  2. Compute the orthogonal projection of \(u=(2,3)\) onto \(v=(1,1)\).
  3. Compute the orthogonal projection of \(u=(1,2,3)\) onto \(v=(2,0,1)\).
  4. What is the length of the orthogonal projection of \(u=(1,2,2)\) onto \(v=(2,1,2)\)?
  5. For \(u=(4,2)\) and \(v=(1,1)\), what is the vector \(u-\operatorname{proj}_v u\)?
  6. If \(v\neq0\), the orthogonal projection of \(u\) onto \(v\) is the zero vector exactly when:
  7. If \(u=c\,v\) with \(c\neq0\), the orthogonal projection of \(u\) onto \(v\) is:
  8. Projecting \(u=(5,3)\) onto \(v=(1,1)\) and onto \(2v=(2,2)\) gives:
  9. What is the minimum distance from the point \(u=(5,0)\) to the line spanned by \(v=(1,2)\)?
  10. If \(\operatorname{proj}_v u=(1,2)\) and \(\operatorname{proj}_v w=(3,-1)\), then \(\operatorname{proj}_v(u+w)\) is:

References

  • Gilbert Strang, Introduction to Linear Algebra, 6th edition, Wellesley-Cambridge Press, 2023. §4.2
  • David C. Lay, Steven R. Lay, Judi J. McDonald, Linear Algebra and Its Applications, 6th edition, Pearson, 2021. §6.2