Section 5.1 Orthogonal Complements and Projections

Definition: 1. If a vector $\overrightarrow{z}$ is orthogonal to every vector in a subspace $W$ of ${\mathbb{R}}^n$ , then $\overrightarrow{z}$ is said to be orthogonal to $W$.2. The set of all vectors $\overrightarrow{z}$ that are orthogonal to $W$ is called the orthogonal complement of $W$ and is denoted by $W^{\mathrm{\perp }}.$

 

Remark: 1. A vector$\overrightarrow{x}$ is in $W^{\mathrm{\perp }}$ if and only if $\overrightarrow{x}\cdot \overrightarrow{y}=0$ for all $\overrightarrow{y}$ in $W$.

 

2. $W^{\mathrm{\perp }}$ is a subspace of ${\mathbb{R}}^n$.

 

3. $(W^{\mathrm{\perp }}{)}^{\mathrm{\perp }}=W$.

 

 

 

Theorem: Let $A$ be an $m\times n$ matrix. The orthogonal complement of the row space of $A$ is the null space of A, and the orthogonal complement of the column space of $A$ is the null space of $A^T$:

$(\text{Row}A{)}^{\mathrm{\perp }}=\text{NulA}$ and $(\text{Col}A{)}^{\mathrm{\perp }}=\text{Nul}{A}^T$.

 

 

Example 1: Find $(\text{Row}A{)}^{\mathrm{\perp }}$ and $(\text{Col}A{)}^{\mathrm{\perp }}$

where $A=\left[\begin{array}{ccc}1& 4& 5\\ 0& -3& 1\\ 0& 6& -2\end{array}\right]$.

 

 

Exercise 1:Find $(\text{Row}A{)}^{\mathrm{\perp }}$ and $(\text{Col}A{)}^{\mathrm{\perp }}$ where $A=\left[\begin{array}{ccc}2& 4& -2\\ 0& -3& 2\\ 2& 1& 0\end{array}\right]$.

 

Example 2: Find $H^{\mathrm{\perp }}$ where $H$ is the plane $\{(a,b,c):3a+b-4c=0\}$.}

 

Exercise 2: Find $H^{\mathrm{\perp }}$ where $H$ is the line
$\{(a,b,c):-a+2b+3c=0,2a+b+c=0\}$.

 

Question: We use an orthogonal basis to get the orthogonal projection of a vector on a subspace. How do we find an orthogonal basis of a subspace?

 

Theorem: The Gram-Schmidt Process

Given a basis $\{\overrightarrow{x_1},...,\overrightarrow{x_p}\}$ for a nonzero subspace $W$ of ${\mathbb{R}}^n$, define $\overrightarrow{v_1}=\overrightarrow{x_1}$, $\overrightarrow{v_2}=\overrightarrow{x_2}-\frac{\overrightarrow{x_2}\cdot \overrightarrow{v_1}}{\overrightarrow{v_1}\cdot \overrightarrow{v_1}}\overrightarrow{v_1}$,…,$\overrightarrow{v_p}=\overrightarrow{x_p}-\sum _{i=1}^{p-1}\frac{\overrightarrow{x_p}\cdot \overrightarrow{v_i}}{\overrightarrow{v_i}\cdot \overrightarrow{v_i}}\overrightarrow{v_i}$. Then $\{\overrightarrow{v_1},...,\overrightarrow{v_p}\}$ is an orthogonal basis for $W$. In addition $\text{Span}\{\overrightarrow{v_1},...,\overrightarrow{v_p}\}=\text{Span}\{\overrightarrow{x_1},...,\overrightarrow{x_p}\}$.

 

Lemma: Let $\{\overrightarrow{x_1},...,\overrightarrow{x_p}\}$ be an orthogonal set in ${\mathbb{R}}^n$. Given $\overrightarrow{u}$ in ${\mathbb{R}}^n$, write $\overrightarrow{x_{p+1}}=\overrightarrow{u}-\frac{\overrightarrow{u}\cdot \overrightarrow{x_1}}{||\overrightarrow{x_1}|{|}^2}\overrightarrow{x_1}-\frac{\overrightarrow{u}\cdot \overrightarrow{x_2}}{||\overrightarrow{x_2}|{|}^2}\overrightarrow{x_2}-\cdots -\frac{\overrightarrow{u}\cdot \overrightarrow{x_p}}{||\overrightarrow{x_p}|{|}^2}\overrightarrow{x_p}$,

then: 

1. $\overrightarrow{x_{p+1}}\cdot \overrightarrow{x_k}=0$ for $k=1,...,p$. 

2. If $\overrightarrow{u}$ is not in $\text{Span}\{\overrightarrow{x_1},...,\overrightarrow{x_p}\}$, then $\overrightarrow{x_{p+1}}\ne 0$ and $\{\overrightarrow{x_1},...,\overrightarrow{x_p},\overrightarrow{x_{p+1}}\}$ is an orthogonal set.

 

 

Example 3: Let $W=\text{Span}\{\overrightarrow{x_1},\overrightarrow{x_2}\}$, where $\overrightarrow{x_1}=\left[\begin{array}{c}1\\ 2\\ 3\end{array}\right]$ and $\overrightarrow{x_2}=\left[\begin{array}{c}-2\\ 0\\ 1\end{array}\right]$.

Construct an orthogonal basis for $W$.

 

Exercise 3: Let $W=\text{Span}\{\overrightarrow{x_1},\overrightarrow{x_2}\}$, where $\overrightarrow{x_1}=\left[\begin{array}{c}1\\ -1\\ 3\end{array}\right]$ and $\overrightarrow{x_2}=\left[\begin{array}{c}2\\ 3\\ 1\end{array}\right]$.

Construct an orthogonal basis for $W$.

 

Example 4: Let $W=\text{Span}\{\overrightarrow{x_1},\overrightarrow{x_2},\overrightarrow{x_3}\}$, where $\overrightarrow{x_1}=\left[\begin{array}{c}1\\ 2\\ 0\\ -1\end{array}\right]$  , $\overrightarrow{x_2}=\left[\begin{array}{c}0\\ 0\\ 1\\ -2\end{array}\right]$ and $\overrightarrow{x_3}=\left[\begin{array}{c}-1\\ 0\\ 1\\ 0\end{array}\right]$.

Construct an orthogonal basis for $W$.}

 

Exercise 4: Let $W=\text{Span}\{\overrightarrow{x_1},\overrightarrow{x_2},\overrightarrow{x_3}\}$, where $\overrightarrow{x_1}=\left[\begin{array}{c}-1\\ 1\\ 1\\ 0\end{array}\right]$  , $\overrightarrow{x_2}=\left[\begin{array}{c}0\\ 1\\ 0\\ -1\end{array}\right]$ and $\overrightarrow{x_3}=\left[\begin{array}{c}2\\ 0\\ 1\\ 2\end{array}\right]$.

Construct an orthogonal basis for $W$.

Remark: To obtain an orthonormal basis from a given basis, one just needs to use the Gram-Schmidt Process to obtain an orthogonal basis then normalize the basis, i.e. divide each vector with its own length to get the unit vector.

 

Example 5: Let $W=\text{Span}\{\overrightarrow{x_1},\overrightarrow{x_2},\overrightarrow{x_3}\}$, where $\overrightarrow{x_1}=\left[\begin{array}{c}1\\ 2\\ 0\\ -1\end{array}\right]$  , $\overrightarrow{x_2}=\left[\begin{array}{c}0\\ 0\\ 1\\ -2\end{array}\right]$ and $\overrightarrow{x_3}=\left[\begin{array}{c}-1\\ 0\\ 1\\ 0\end{array}\right]$.

Construct an orthonormal basis for $W$.

 

 

Exercise 5: Let $W=\text{Span}\{\overrightarrow{x_1},\overrightarrow{x_2},\overrightarrow{x_3}\}$ , where $\overrightarrow{x_1}=\left[\begin{array}{c}-1\\ 1\\ 1\\ 0\end{array}\right]$ , $\overrightarrow{x_2}=\left[\begin{array}{c}0\\ 1\\ 0\\ -1\end{array}\right]$  and $\overrightarrow{x_3}=\left[\begin{array}{c}2\\ 0\\ 1\\ 2\end{array}\right]$.

Construct an orthonormal basis for $W$.

 

GroupWork 1: True or False. All vectors and subspaces are in ${\mathbb{R}}^n$.

a. If $\overrightarrow{z}$ is orthogonal to $\overrightarrow{u_1}$ and $\overrightarrow{u_2}$ and if $W=\text{Span}\{\overrightarrow{u_1},\overrightarrow{u_2}\}$, then $\overrightarrow{z}$ must be in $W^{\mathrm{\perp }}$.

 

b. For an $m\times n$ matrix $A$, vectors in the null space of $A$ are orthogonal to vectors in the row space of $A$.

 

c. For a square matrix $A$, vectors in Col$A$ are orthogonal to vectors in Nul$A$.

 

d. If $\overrightarrow{x}$ is orthogonal to every vector in a subspace $W$ then $\overrightarrow{x}$ is in $W^{\mathrm{\perp }}$.

 

e. For each $\overrightarrow{y}$ and each subspace $W$, the vector $\overrightarrow{y}-{\text{proj}}_W\overrightarrow{y}$ is orthogonal to $W$.

 

f. The orthogonal projection $\hat{y}$ of $\overrightarrow{y}$ onto a subspace $W$ can sometimes depend on the orthogonal basis for $W$ used to compute $\hat{y}$.

 

g. If $\overrightarrow{y}$ is in a subspace $W$, then the orthogonal projection of $\overrightarrow{y}$ onto $W$ is $\overrightarrow{y}$ itself.

 

GroupWork 2: Let $W$ be a subspace of ${\mathbb{R}}^n$ with an orthogonal basis $\{\overrightarrow{w_1},...,\overrightarrow{w_p}\}$, and let $\{\overrightarrow{v_1},...,\overrightarrow{v_q}\}$ be an orthogonal basis for $W^{\mathrm{\perp }}$.

(1) Show that $\{\overrightarrow{w_1},...,\overrightarrow{w_p},\overrightarrow{v_1},...,\overrightarrow{v_q}\}$ is an orthogonal basis of ${\mathbb{R}}^n$.

(2) Show that $\text{dim}W+\text{dim}{W}^{\mathrm{\perp }}=n$.

 

GroupWork 3: True or False. All vectors and subspaces are in ${\mathbb{R}}^n$.

a. If $W$ is a subspace of ${\mathbb{R}}^n$ and if $\overrightarrow{v}$ is in both $W$ and $W^{\mathrm{\perp }}$ then $\overrightarrow{v}$ must be the zero vector.

 

b. If $\overrightarrow{y}=\overrightarrow{z_1}+\overrightarrow{z_2}$, where $\overrightarrow{z_1}$ is in subspace $W$ and $\overrightarrow{z_2}$ is in subspace $W^{\mathrm{\perp }},$ then $\overrightarrow{z_1}$ must be the orthogonal projection of $\overrightarrow{y}$ onto $W$.

 

c. If $\{\overrightarrow{v_1},\overrightarrow{v_2},\overrightarrow{v_3}\}$ is an orthogonal basis for $W$, then multiplying $\overrightarrow{v_3}$ by a scalar $c$ gives a new orthogonal basis $\{\overrightarrow{v_1},\overrightarrow{v_2},c\overrightarrow{v_3}\}$.

 

d. If $W=\text{Span}\{\overrightarrow{v_1},\overrightarrow{v_2},\overrightarrow{v_3}\}$, and if $\{\overrightarrow{v_1},\overrightarrow{v_2},\overrightarrow{v_3}\}$ is an orthogonal set in $W$, then $\{\overrightarrow{v_1},\overrightarrow{v_2},\overrightarrow{v_3}\}$ is a basis for $W$.

 

e. If $\overrightarrow{x}$ is not in a subspace $W$, then $\overrightarrow{x}-{\text{proj}}_W\overrightarrow{x}$ is not zero.