# Section 5.1 Orthogonal Complements and Projections

Definition: 1. If a vector$\overrightarrow{z}$ is orthogonal to every vector in a subspace $W$ of $\mathbb{R}^{n}$ , then $\overrightarrow{z}$ is said to be orthogonal to $W$.

2. The set of all vectors$\overrightarrow{z}$ that are orthogonal to $W$ is called the orthogonal complement of $W$ and is denoted by $W^{\bot}.$

Remark: 1. A vector$\overrightarrow{x}$ is in $W^{\bot}$ if and only if $\overrightarrow{x}\cdot\overrightarrow{y}=0$ for all $\overrightarrow{y}$ in $W$.

2. $W^{\bot}$ is a subspace of $\mathbb{R}^{n}$.

3. $(W^{\bot})^{\bot}=W$.

Theorem: Let$A$ be an $m\times n$ matrix. The orthogonal complement of the row space of $A$ is the null space of A, and the orthogonal complement of the column space of $A$ is the null space of $A^{T}$:

$(\mbox{Row}A)^{\bot}=\mbox{Nul}{A}$ and $(\mbox{Col}A)^{\bot}=\mbox{Nul}A^{T}$.

Example 1: Find $(\mbox{Row}A)^{\bot}$ and $(\mbox{Col}A)^{\bot}$

where $A=\left[\begin{array}{ccc} 1 & 4 & 5\\ 0 & -3 & 1\\ 0 & 6 & -2 \end{array}\right]$.

Exercise 1: Find$(\mbox{Row}A)^{\bot}$ and $(\mbox{Col}A)^{\bot}$ where $A=\left[\begin{array}{ccc} 2 & 4 & -2\\ 0 & -3 & 2\\ 2 & 1 & 0 \end{array}\right]$.

Example 2: Find $H^{\bot}$ where $H$ is the plane $\{(a,b,c):3a+b-4c=0\}$.}

Exercise 2: Find $H^{\bot}$ where $H$ is the line
$\{(a,b,c):-a+2b+3c=0,\mbox{ }2a+b+c=0\}$.

Question: We use an orthogonal basis to get the orthogonal projection of a vector on a subspace. How do we find an orthogonal basis of a subspace?

Theorem: The Gram-Schmidt Process

Given a basis$\{\overrightarrow{x_{1}},...,\overrightarrow{x_{p}}\}$ for a nonzero subspace $W$ of $\mathbb{R}^{n}$, define $\overrightarrow{v_{1}}=\overrightarrow{x_{1}}$, $\overrightarrow{v_{2}}=\overrightarrow{x_{2}}-\frac{\overrightarrow{x_{2}}\cdot\overrightarrow{v_{1}}}{\overrightarrow{v_{1}}\cdot\overrightarrow{v_{1}}}\overrightarrow{v_{1}}$,…,$\overrightarrow{v_{p}}=\overrightarrow{x_{p}}-\sum_{i=1}^{p-1}\frac{\overrightarrow{x_{p}}\cdot\overrightarrow{v_{i}}}{\overrightarrow{v_{i}}\cdot\overrightarrow{v_{i}}}\overrightarrow{v_{i}}$. Then$\{\overrightarrow{v_{1}},...,\overrightarrow{v_{p}}\}$ is an orthogonal basis for $W$. In addition $\mbox{Span}\{\overrightarrow{v_{1}},...,\overrightarrow{v_{p}}\}=\mbox{Span}\{\overrightarrow{x_{1}},...,\overrightarrow{x_{p}}\}$.

Lemma: Let $\{\overrightarrow{x_{1}},...,\overrightarrow{x_{p}}\}$ be an orthogonal set in $\mathbb{R}^{n}$. Given $\overrightarrow{u}$ in $\mathbb{R}^{n}$, write

$\overrightarrow{x_{p+1}}=\overrightarrow{u}-\frac{\overrightarrow{u}\cdot\overrightarrow{x_{1}}}{||\overrightarrow{x_{1}}||^{2}}\overrightarrow{x_{1}}-\frac{\overrightarrow{u}\cdot\overrightarrow{x_{2}}}{||\overrightarrow{x_{2}}||^{2}}\overrightarrow{x_{2}}-\cdots-\frac{\overrightarrow{u}\cdot\overrightarrow{x_{p}}}{||\overrightarrow{x_{p}}||^{2}}\overrightarrow{x_{p}}$,

then:

1. $\overrightarrow{x_{p+1}}\cdot\overrightarrow{x_{k}}=0$ for $k=1,...,p$.

2. If $\overrightarrow{u}$ is not in $\text{Span}\{\overrightarrow{x_{1}},...,\overrightarrow{x_{p}}\}$, then $\overrightarrow{x_{p+1}}\neq0$ and $\{\overrightarrow{x_{1}},...,\overrightarrow{x_{p}},\overrightarrow{x_{p+1}}\}$ is an orthogonal set.

Example 3: Let $W=\mbox{Span}\{\overrightarrow{x_{1}},\overrightarrow{x_{2}}\}$, where $\overrightarrow{x_{1}}=\left[\begin{array}{c} 1\\ 2\\ 3 \end{array}\right]$ and$\overrightarrow{x_{2}}=\left[\begin{array}{c} -2\\ 0\\ 1 \end{array}\right]$.

Construct an orthogonal basis for $W$.

Exercise 3: Let$W=\mbox{Span}\{\overrightarrow{x_{1}},\overrightarrow{x_{2}}\}$, where $\overrightarrow{x_{1}}=\left[\begin{array}{c} 1\\ -1\\ 3 \end{array}\right]$ and $\overrightarrow{x_{2}}=\left[\begin{array}{c} 2\\ 3\\ 1 \end{array}\right]$.

Construct an orthogonal basis for $W$.

Example 4: Let $W=\mbox{Span}\{\overrightarrow{x_{1}},\overrightarrow{x_{2}},\overrightarrow{x_{3}}\}$, where $\overrightarrow{x_{1}}=\left[\begin{array}{c} 1\\ 2\\ 0\\ -1 \end{array}\right]$  , $\overrightarrow{x_{2}}=\left[\begin{array}{c} 0\\ 0\\ 1\\ -2 \end{array}\right]$ and $\overrightarrow{x_{3}}=\left[\begin{array}{c} -1\\ 0\\ 1\\ 0 \end{array}\right]$.

Construct an orthogonal basis for $W$.

Exercise 4: Let $W=\mbox{Span}\{\overrightarrow{x_{1}},\overrightarrow{x_{2}},\overrightarrow{x_{3}}\}$, where $\overrightarrow{x_{1}}=\left[\begin{array}{c} -1\\ 1\\ 1\\ 0 \end{array}\right]$  , $\overrightarrow{x_{2}}=\left[\begin{array}{c} 0\\ 1\\ 0\\ -1 \end{array}\right]$ and $\overrightarrow{x_{3}}=\left[\begin{array}{c} 2\\ 0\\ 1\\ 2 \end{array}\right]$.

Construct an orthogonal basis for $W$.

Remark: To obtain an orthonormal basis from a given basis, one just need to use the Gram-Schmidt Process to obtain an orthogonal basis then normalize the basis, i.e. divide each vector with its own length to get the unit vector.

Example 5: Let $W=\mbox{Span}\{\overrightarrow{x_{1}},\overrightarrow{x_{2}},\overrightarrow{x_{3}}\}$, where $\overrightarrow{x_{1}}=\left[\begin{array}{c} 1\\ 2\\ 0\\ -1 \end{array}\right]$  , $\overrightarrow{x_{2}}=\left[\begin{array}{c} 0\\ 0\\ 1\\ -2 \end{array}\right]$ and$\overrightarrow{x_{3}}=\left[\begin{array}{c} -1\\ 0\\ 1\\ 0 \end{array}\right]$.

Construct an orthonormal basis for $W$.

Exercise 5: Let $W=\mbox{Span}\{\overrightarrow{x_{1}},\overrightarrow{x_{2}},\overrightarrow{x_{3}}\}$ , where $\overrightarrow{x_{1}}=\left[\begin{array}{c} -1\\ 1\\ 1\\ 0 \end{array}\right]$ ,$\overrightarrow{x_{2}}=\left[\begin{array}{c} 0\\ 1\\ 0\\ -1 \end{array}\right]$  and $\overrightarrow{x_{3}}=\left[\begin{array}{c} 2\\ 0\\ 1\\ 2 \end{array}\right]$.

Construct an orthonormal basis for $W$.

Group Work 1: True or False. All vectors and subspaces are in $\mathbb{R}^{n}$.

a. If$\overrightarrow{z}$ is orthogonal to $\overrightarrow{u_{1}}$ and$\overrightarrow{u_{2}}$ and if $W=\mbox{Span}\{\overrightarrow{u_{1}},\overrightarrow{u_{2}}\}$, then $\overrightarrow{z}$ must be in $W^{\bot}$.

b. For an $m\times n$ matrix $A$, vectors in the null space of $A$ are orthogonal to vectors in the row space of$A$.

c. For a square matrix $A$, vectors in Col$A$ are orthogonal to vectors in Nul$A$.

d. If $\overrightarrow{x}$ is orthogonal to every vector in a subspace $W$ then $\overrightarrow{x}$ is in $W^{\bot}$.

e. For each $\overrightarrow{y}$ and each subspace $W$, the vector $\overrightarrow{y}-\mbox{proj}_{W}\overrightarrow{y}$ is orthogonal to $W$.

f. The orthogonal projection $\widehat{y}$ of $\overrightarrow{y}$ onto a subspace $W$ can sometimes depend on the orthogonal basis for $W$ used to compute $\widehat{y}$.

g. If $\overrightarrow{y}$ is in a subspace $W$, then the orthogonal projection of $\overrightarrow{y}$ onto $W$ is $\overrightarrow{y}$ itself.

Group Work 2: Let $A$ be an $m\times n$ matrix. Show that every vector $\overrightarrow{x}$ in $\mathbb{R}^{n}$ can be uniquely written in the form of $\overrightarrow{x}=\overrightarrow{r}+\overrightarrow{u}$ where $\overrightarrow{r}$ is in Row$A$ and $\overrightarrow{u}$ is in Nul$A$. Use this to show if $A\overrightarrow{x}=\overrightarrow{b}$ has a unique solution then there is a unique $\overrightarrow{r}$ in Row$A$ such that $A\overrightarrow{r}=\overrightarrow{b}$.

Group Work 3: True or False. All vectors and subspaces are in $\mathbb{R}^{n}$.

a. If $W$ is a subspace of $\mathbb{R}^{n}$ and if $\overrightarrow{v}$ is in both $W$ and$W^{\bot}$ then$\overrightarrow{v}$ must be the zero vector.

b. If $\overrightarrow{y}=\overrightarrow{z_{1}}+\overrightarrow{z_{2}}$, where $\overrightarrow{z_{1}}$ is in subspace $W$ and $\overrightarrow{z_{2}}$ is in subspace $W^{\bot},$ then $\overrightarrow{z_{1}}$ must be the orthogonal projection of$\overrightarrow{y}$ onto $W$.

c. If $\{\overrightarrow{v_{1}},\overrightarrow{v_{2}},\overrightarrow{v_{3}}\}$ is an orthogonal basis for $W$, then multiplying $\overrightarrow{v_{3}}$ by a scalar $c$ gives a new orthogonal basis $\{\overrightarrow{v_{1}},\overrightarrow{v_{2}},c\overrightarrow{v_{3}}\}$.

d. If $W=\mbox{Span}\{\overrightarrow{x_{1}},\overrightarrow{x_{2}},\overrightarrow{x_{3}}\}$ with$\{\overrightarrow{x_{1}},\overrightarrow{x_{2}},\overrightarrow{x_{3}}\}$ linearly independent, and if$\{\overrightarrow{v_{1}},\overrightarrow{v_{2}},\overrightarrow{v_{3}}\}$ is an orthogonal set in $W$, then $\{\overrightarrow{v_{1}},\overrightarrow{v_{2}},\overrightarrow{v_{3}}\}$ is a basis for $W$.

e. If $\overrightarrow{x}$ is not in a subspace $W$, then $\overrightarrow{x}-\mbox{proj}_{W}\overrightarrow{x}$ is not zero.

Group Work 4: Let $W$ be a subspace of $\mathbb{R}^{n}$ with an orthogonal basis $\{\overrightarrow{w_{1}},...,\overrightarrow{w_{p}}\}$, and let $\{\overrightarrow{v_{1}},...,\overrightarrow{v_{q}}\}$ be an orthogonal basis for $W^{\bot}$.

(1) Show that $\{\overrightarrow{w_{1}},...,\overrightarrow{w_{p}},\overrightarrow{v_{1}},...,\overrightarrow{v_{q}}\}$ is an orthogonal basis of $\mathbb{R}^{n}$.

(2) Show that $\mbox{dim}W+\mbox{dim}W^{\bot}=n$.