lec11.wb

@course{MATH 1030}
@setchapter{11}
@chapter{Spanning Sets}
The lecture is based on Beezer, A first course in Linear algebra. Ver 3.5 Downloadable at
@href{http://linear.ups.edu/download.html} .

The print version can be downloaded at
@href{http://linear.ups.edu/download/fcla-3.50-print.pdf} .

<h5 class="notkw">Reference.</h5><ul>
<li> Beezer, Ver 3.5 Section SS (print version p83 - p94) </li>
<li> Strang, Sect 2.3 </li></ul>

<h5 class="notkw">Exercise.</h5><ul>
<li> Exercises with solutions can be downloaded at @href{http://linear.ups.edu/download/fcla-3.50-solution-manual.pdf} Section SS (p.34-40) C40-45, C50, C60, M10, M11, M12. (Replace $\mathbb{C}$ by ${\mathbb{R}}^{\hbox{}}$ in the following questions) T10, T20, T21, T22. </li>
<li> Strang, Sect 2.3. </li></ul>
@slide
In this section we will provide an extremely compact way to describe an infinite set of vectors, making use of linear combinations. This will give us a convenient way to describe the solution set of a linear system, the null space of a matrix, and many other sets of vectors.
@section{Span of a Set of Vectors}
@label{SSV}
<!--
We saw that the solution set of a homogeneous system can be described as all possible linear combinations of two particular sets of vectors. This is a useful way to construct or describe infinite sets of vectors, so we encapsulate the idea in a definition.
-->

@defn
@title{Span of a Set of Column Vectors}
@label{SSCV}
Given a set of vectors
\begin{align*}
\displaystyle S=\{\mathbf{u}_{1},\,\mathbf{u}_{2},\,\mathbf{u}_{3},\,\ldots,\,\mathbf{u}_{n}\},
\end{align*}
their @keyword{span}, $\mathrm{Span}\,S$, is the set of all linear combinations of $\mathbf{u}_{1},\,\mathbf{u}_{2},\,\mathbf{u}_{3},\,\ldots,\,\mathbf{u}_{n}$.
Symbolically,
\begin{align*}
\displaystyle
\mathrm{Span}\,S&\displaystyle=\left\{\left.\alpha_{1}\mathbf{u}_{1}+\alpha_{2}\mathbf{u}_{2}+\alpha_{3}\mathbf{u}_{3}+\cdots+\alpha_{n}\mathbf{u}_{n}\,\right|\,\alpha_{i}\in{\mathbb{R}}^{\hbox{}},\,1\leq i\leq n\right\} \\
&\displaystyle=\left\{\left.\sum_{i=1}^{n}\alpha_{i}\mathbf{u}_{i}\,\right|\,\alpha_{i}\in{\mathbb{R}}^{\hbox{}},\,1\leq i\leq n\right\}
\end{align*}
@end
@slide
@thm
Let $S=\{\mathbf{u}_{1},\ldots,\mathbf{u}_{k}\}\subseteq V={\mathbb{R}}^{m}$.
Then $\mathrm{Span}\,S$ is a subspace of $V$.
@end
@proof
@newcol
Obviously $\mathrm{Span}\,S$ is nonempty. Let $\alpha\in{\mathbb{R}}^{\hbox{}}$, $\mathbf{v},\mathbf{w}\in W=\mathrm{Span}\,S$.
Then there exists $\alpha_{1},\ldots,\alpha_{k},\beta_{1},\ldots,\beta_{k}$ such that
\begin{align*}
\displaystyle \mathbf{v}=\alpha_{1}\mathbf{u}_{1}+\cdots+\alpha_{k}\mathbf{u}_{k},
\end{align*}
\begin{align*}
\displaystyle \mathbf{w}=\beta_{1}\mathbf{u}_{1}+\cdots+\beta_{k}\mathbf{u}_{k}.
\end{align*}
Then
\begin{align*}
\displaystyle \alpha\mathbf{v}+\mathbf{w}=(\alpha\alpha_{1}+\beta_{1})\mathbf{u}_{1}+\cdots+(\alpha\alpha_{k}+\beta_{k})\mathbf{u}_{k}
\end{align*}
is in $\mathrm{Span}\,S$.
Thus by @ref{b97161010780aa4b703cb89b1294f822}, $W$ is a subspace.
@qed
@endcol
@end
<strong>Main Questions.</strong>
@newcol
<ol class="ltx_enumerate">
<li class="ltx_item"> Determine if a vector $\mathbf{v}$ is an element of $\mathrm{Span}\,S$.</li>
<li class="ltx_item"> Describe the set $\mathrm{Span}\,S$.</li>
<li class="ltx_item"> Is $\mathrm{Span}\,S$ equal to ${\mathbb{R}}^{m}$?</li></ol>
@endcol
@slide
@eg
@label{ABS}
Consider the following set of 5 vectors, $S$, from ${\mathbb{R}}^{4}$:
\begin{align*}
\displaystyle S=\left\{\begin{bmatrix}1\\
1\\
3\\
1\end{bmatrix},\,\begin{bmatrix}2\\
1\\
2\\
-1\end{bmatrix},\,\begin{bmatrix}7\\
3\\
5\\
-5\end{bmatrix},\,\begin{bmatrix}1\\
1\\
-1\\
2\end{bmatrix},\,\begin{bmatrix}-1\\
0\\
9\\
0\end{bmatrix}\right\}.
\end{align*}
@newcol
Consider the infinite set of vectors $\mathrm{Span}\,S$ formed by all linear combinations of the elements of $S$. Here are four vectors which we definitely know are elements of $\mathrm{Span}\,S$:
\begin{align*}
\displaystyle \mathbf{w}=(2)\begin{bmatrix}1\\
1\\
3\\
1\end{bmatrix}+(1)\begin{bmatrix}2\\
1\\
2\\
-1\end{bmatrix}+(-1)\begin{bmatrix}7\\
3\\
5\\
-5\end{bmatrix}+(2)\begin{bmatrix}1\\
1\\
-1\\
2\end{bmatrix}+(3)\begin{bmatrix}-1\\
0\\
9\\
0\end{bmatrix}=\begin{bmatrix}-4\\
2\\
28\\
10\end{bmatrix}
\end{align*}
\begin{align*}
\displaystyle \mathbf{x}=(5)\begin{bmatrix}1\\
1\\
3\\
1\end{bmatrix}+(-6)\begin{bmatrix}2\\
1\\
2\\
-1\end{bmatrix}+(-3)\begin{bmatrix}7\\
3\\
5\\
-5\end{bmatrix}+(4)\begin{bmatrix}1\\
1\\
-1\\
2\end{bmatrix}+(2)\begin{bmatrix}-1\\
0\\
9\\
0\end{bmatrix}=\begin{bmatrix}-26\\
-6\\
2\\
34\end{bmatrix}
\end{align*}
\begin{align*}
\displaystyle \mathbf{y}=(1)\begin{bmatrix}1\\
1\\
3\\
1\end{bmatrix}+(0)\begin{bmatrix}2\\
1\\
2\\
-1\end{bmatrix}+(1)\begin{bmatrix}7\\
3\\
5\\
-5\end{bmatrix}+(0)\begin{bmatrix}1\\
1\\
-1\\
2\end{bmatrix}+(1)\begin{bmatrix}-1\\
0\\
9\\
0\end{bmatrix}=\begin{bmatrix}7\\
4\\
17\\
-4\end{bmatrix}
\end{align*}
\begin{align*}
\displaystyle \mathbf{z}=(0)\begin{bmatrix}1\\
1\\
3\\
1\end{bmatrix}+(0)\begin{bmatrix}2\\
1\\
2\\
-1\end{bmatrix}+(0)\begin{bmatrix}7\\
3\\
5\\
-5\end{bmatrix}+(0)\begin{bmatrix}1\\
1\\
-1\\
2\end{bmatrix}+(0)\begin{bmatrix}-1\\
0\\
9\\
0\end{bmatrix}=\begin{bmatrix}0\\
0\\
0\\
0\end{bmatrix}.
\end{align*}
@col
<strong>Fundamental question:</strong> Determine if a given vector is an element of the set or not. Let us learn more about $\mathrm{Span}\,S$ by investigating which vectors are elements of the set, and which are not.

First, is $\mathbf{u}=\begin{bmatrix}-15\\
-6\\
19\\
5\end{bmatrix}$ an element of $\mathrm{Span}\,S$?

@col
In other words, are there scalars $\alpha_{1},\,\alpha_{2},\,\alpha_{3},\,\alpha_{4},\,\alpha_{5}$ such that:
\begin{align*}
\displaystyle \alpha_{1}\begin{bmatrix}1\\
1\\
3\\
1\end{bmatrix}+\alpha_{2}\begin{bmatrix}2\\
1\\
2\\
-1\end{bmatrix}+\alpha_{3}\begin{bmatrix}7\\
3\\
5\\
-5\end{bmatrix}+\alpha_{4}\begin{bmatrix}1\\
1\\
-1\\
2\end{bmatrix}+\alpha_{5}\begin{bmatrix}-1\\
0\\
9\\
0\end{bmatrix}=\mathbf{u}=\begin{bmatrix}-15\\
-6\\
19\\
5\end{bmatrix}?
\end{align*}
@col
Searching for such scalars is equivalent to finding a solution to the linear system of equations with augmented matrix:
\begin{align*}
\displaystyle \left[\begin{array}[]{ccccc|c}1&2&7&1&-1&-15\\
1&1&3&1&0&-6\\
3&2&5&-1&9&19\\
1&-1&-5&2&0&5\end{array}\right].
\end{align*}
@col
This matrix row-reduces to
\begin{align*}
\displaystyle \left[\begin{array}[]{ccccc|c}\boxed{1}&0&-1&0&3&10\\
0&\boxed{1}&4&0&-1&-9\\
0&0&0&\boxed{1}&-2&-7\\
0&0&0&0&0&0\end{array}\right].
\end{align*}
@col
At this point, we see that the system is consistent, so we know there is a solution for the five scalars $\alpha_{1},\,\alpha_{2},\,\alpha_{3},\,\alpha_{4},\,\alpha_{5}$. This is enough evidence for us to say that $\mathbf{u}\in\mathrm{Span}\,S$.

Moreover, we can compute an actual solution, for example:

@col
\begin{align*}
\displaystyle\alpha_{1}&\displaystyle=2&\displaystyle\alpha_{2}&\displaystyle=1&\displaystyle\alpha_{3}&\displaystyle=-2&\displaystyle\alpha_{4}&\displaystyle=-3&\displaystyle\alpha_{5}&\displaystyle=2.
\end{align*}
This particular solution allows us to write
\begin{align*}
\displaystyle (2)\begin{bmatrix}1\\
1\\
3\\
1\end{bmatrix}+(1)\begin{bmatrix}2\\
1\\
2\\
-1\end{bmatrix}+(-2)\begin{bmatrix}7\\
3\\
5\\
-5\end{bmatrix}+(-3)\begin{bmatrix}1\\
1\\
-1\\
2\end{bmatrix}+(2)\begin{bmatrix}-1\\
0\\
9\\
0\end{bmatrix}=\mathbf{u}=\begin{bmatrix}-15\\
-6\\
19\\
5\end{bmatrix}
\end{align*}
making it even more obvious that $\mathbf{u}\in\mathrm{Span}\,S$.

@col
We now determine if $\mathbf{v}=\begin{bmatrix}3\\
1\\
2\\
-1\end{bmatrix}$ an element of $\mathrm{Span}\,S$.

@col
We want to know if there are scalars $\alpha_{1},\,\alpha_{2},\,\alpha_{3},\,\alpha_{4},\,\alpha_{5}$ such that:
\begin{align*}
\displaystyle \alpha_{1}\begin{bmatrix}1\\
1\\
3\\
1\end{bmatrix}+\alpha_{2}\begin{bmatrix}2\\
1\\
2\\
-1\end{bmatrix}+\alpha_{3}\begin{bmatrix}7\\
3\\
5\\
-5\end{bmatrix}+\alpha_{4}\begin{bmatrix}1\\
1\\
-1\\
2\end{bmatrix}+\alpha_{5}\begin{bmatrix}-1\\
0\\
9\\
0\end{bmatrix}=\mathbf{v}=\begin{bmatrix}3\\
1\\
2\\
-1\end{bmatrix}
\end{align*}
@col
Again, this is equivalent to finding a solution to the linear system of equations with augmented matrix:
\begin{align*}
\displaystyle \left[\begin{array}[]{ccccc|c}1&2&7&1&-1&3\\
1&1&3&1&0&1\\
3&2&5&-1&9&2\\
1&-1&-5&2&0&-1\end{array}\right].
\end{align*}
This matrix row-reduces to
\begin{align*}
\displaystyle \left[\begin{array}[]{ccccc|c}\boxed{1}&0&-1&0&3&0\\
0&\boxed{1}&4&0&-1&0\\
0&0&0&\boxed{1}&-2&0\\
0&0&0&0&0&\boxed{1}\end{array}\right].
\end{align*}
@col
At this point, we see that the system is inconsistent, so we know there is no solution for the five scalars $\alpha_{1},\,\alpha_{2},\,\alpha_{3},\,\alpha_{4},\,\alpha_{5}$. This is enough evidence for us to say that $\mathbf{v}\not\in\mathrm{Span}\,S$. End of story.
@endcol
@end
@slide
From the previous example, we have the following theorem:
@thm
Suppose that $\mathbf{u}_{1},\,\mathbf{u}_{2},\,\mathbf{u}_{3},\,\ldots,\,\mathbf{u}_{n}$ are in $\mathbf{R}^{m}$.
Let $A$ be the $m\times n$ matrix whose $i$-th column is $\mathbf{u}_{i}$.
Then $\mathbf{v}\in\mathrm{Span}\,S$ if and only if $\mathcal{LS}({A},{\mathbf{v}})$ is consistent.
@end
@slide
@skip
@eg
<strong>Computational Technique.</strong>

Given $S=\{\mathbf{u}_{1},\,\mathbf{u}_{2},\,\mathbf{u}_{3},\,\ldots,\,\mathbf{u}_{n}\}$ in ${\mathbb{R}}^{m}$, determine if
\begin{align*}
\displaystyle \mathbf{v}=\begin{bmatrix}v_{1}\\
\vdots\\
v_{m}\end{bmatrix}\in\mathrm{Span}\,S.
\end{align*}
@end
@sol
@enumerate
@item
@newcol
To determine if $\mathbf{v}\in\mathrm{Span}\,S$, we need to find $\alpha_{1},\ldots,\alpha_{n}$ such that
\begin{align*}
\displaystyle \alpha_{1}\mathbf{u}_{1}+\cdots+\alpha_{n}\mathbf{u}_{n}=\mathbf{v}.
\end{align*}
@endcol
@item
@newcol
This is equivalent to solving the system of linear equations $\mathcal{LS}({A},{\mathbf{v}})$,
where $A$ is the $m\times n$ matrix whose $i$-th column is $\mathbf{u}_{i}$.
@endcol
@item
@newcol
Row-reduce the augmented matrix $[A|\mathbf{v}]$ to an RREF $B$.
@enumerate
@item
@newcol
If the last column of $B$ is a pivot column, then the system is inconsistent and $\mathbf{v}\notin\mathrm{Span}\,S$.
@endcol
@item
@newcol
If the last column of $B$ is not a pivot column, then the system is consistent and $\mathbf{v}\in\mathrm{Span}\,S$.
@endcol
@endenumerate
@endcol
@endenumerate
@end
@slide
@skip
@eg
@label{SCAA}
Consider:
\begin{align*}
\displaystyle S=\{\mathbf{u}_{1},\,\mathbf{u}_{2},\,\mathbf{u}_{3}\}=\left\{\begin{bmatrix}1\\
2\\
1\end{bmatrix},\,\begin{bmatrix}-1\\
1\\
1\end{bmatrix},\,\begin{bmatrix}2\\
1\\
0\end{bmatrix}\right\}
\end{align*}
and consider the infinite set $\mathrm{Span}\,S$.

<!-- First, as an example, note that
\begin{align*}
\displaystyle \mathbf{v}=(5)\begin{bmatrix}1\\
2\\
1\end{bmatrix}+(-3)\begin{bmatrix}-1\\
1\\
1\end{bmatrix}+(7)\begin{bmatrix}2\\
1\\
0\end{bmatrix}=\begin{bmatrix}22\\
14\\
2\end{bmatrix}
\end{align*}
is in $\mathrm{Span}\,S$, since it is a linear combination of $\mathbf{u}_{1},\,\mathbf{u}_{2},\,\mathbf{u}_{3}$. There is nothing magical about the scalars $\alpha_{1}=5,\,\alpha_{2}=-3,\,\alpha_{3}=7$; they can be chosen arbitrarily. So repeat this part of the example yourself, using different values of $\alpha_{1},\,\alpha_{2},\,\alpha_{3}$. What happens if you choose all three scalars to be zero?
So we know how to quickly construct sample elements of the set $\mathrm{Span}\,S$. A slightly different question arises when you are handed a vector of the correct size and asked if it is an element of $\mathrm{Span}\,S$. For example, is -->



Does $\mathbf{w}=\begin{bmatrix}1\\
8\\
5\end{bmatrix}$ lie in $\mathrm{Span}\,S$?

@newcol
To answer this question, we will look for scalars $\alpha_{1},\,\alpha_{2},\,\alpha_{3}$ such that
\begin{align*}
\displaystyle\alpha_{1}\mathbf{u}_{1}+\alpha_{2}\mathbf{u}_{2}+\alpha_{3}\mathbf{u}_{3}&\displaystyle=\mathbf{w}.
\end{align*}
This is equivalent to solving the system of linear equations
\begin{align*}
\displaystyle\alpha_{1}-\alpha_{2}+2\alpha_{3}&\displaystyle=1 \\
\displaystyle 2\alpha_{1}+\alpha_{2}+\alpha_{3}&\displaystyle=8 \\
\displaystyle\alpha_{1}+\alpha_{2}&\displaystyle=5.
\end{align*}
Building the augmented matrix for this linear system, and row-reducing, gives:

@col
\begin{align*}
\displaystyle \left[\begin{array}[]{ccc|c}\boxed{1}&0&1&3\\
0&\boxed{1}&-1&2\\
0&0&0&0\end{array}\right].
\end{align*}
This system has infinitely many solutions (there is a free variable in $x_{3}$), but all we need is one solution vector. The solution,
\begin{align*}
\displaystyle\alpha_{1}&\displaystyle=2&\displaystyle\alpha_{2}&\displaystyle=3&\displaystyle\alpha_{3}&\displaystyle=1
\end{align*}
tells us that:

@col
\begin{align*}
\displaystyle (2)\mathbf{u}_{1}+(3)\mathbf{u}_{2}+(1)\mathbf{u}_{3}=\mathbf{w}.
\end{align*}
@col
So we are convinced that $\mathbf{w}$ really is in $\mathrm{Span}\,S$.

@col
Notice that there is an infinite number of ways to answer this question affirmatively. We could choose a different solution, this time choosing the free variable to be zero,
\begin{align*}
\displaystyle\alpha_{1}&\displaystyle=3&\displaystyle\alpha_{2}&\displaystyle=2&\displaystyle\alpha_{3}&\displaystyle=0,
\end{align*}
showing that
\begin{align*}
\displaystyle (3)\mathbf{u}_{1}+(2)\mathbf{u}_{2}+(0)\mathbf{u}_{3}=\mathbf{w}.
\end{align*}
Verifying the arithmetic in this second solution will make it obvious that $\mathbf{w}$ is in this span. And of course, we now realize that there are an <em>infinite</em> number of ways to realize $\mathbf{w}$ as element of $\mathrm{Span}\,S$.

@col
Let us ask the same type of question again, but this time with: $\mathbf{y}=\begin{bmatrix}2\\
4\\
3\end{bmatrix}$.

Is $\mathbf{y}\in\mathrm{Span}\,S$?

@col
So we will look for scalars $\alpha_{1},\,\alpha_{2},\,\alpha_{3}$ such that
\begin{align*}
\displaystyle\alpha_{1}\mathbf{u}_{1}+\alpha_{2}\mathbf{u}_{2}+\alpha_{3}\mathbf{u}_{3}&\displaystyle=\mathbf{y}.
\end{align*}
This is equivalent to finding solutions to the system of equations
\begin{align*}
\displaystyle\alpha_{1}-\alpha_{2}+2\alpha_{3}&\displaystyle=2 \\
\displaystyle 2\alpha_{1}+\alpha_{2}+\alpha_{3}&\displaystyle=4 \\
\displaystyle\alpha_{1}+\alpha_{2}&\displaystyle=3,
\end{align*}
Building the augmented matrix for this linear system and row-reducing gives
\begin{align*}
\displaystyle \left[\begin{array}[]{ccc|c}\boxed{1}&0&1&0\\
0&\boxed{1}&-1&0\\
0&0&0&\boxed{1}\end{array}\right].
\end{align*}
This system is inconsistent because the last column is a pivot column.
So there are no scalars $\alpha_{1},\,\alpha_{2},\,\alpha_{3}$ that will create a linear combination of $\mathbf{u}_{1},\,\mathbf{u}_{2},\,\mathbf{u}_{3}$ that equals $\mathbf{y}$. More precisely, $\mathbf{y}\not\in\mathrm{Span}\,S$.

There are three things to observe in this example. <ol class="ltx_enumerate">
<li class="ltx_item"> It is easy to construct vectors in $\mathrm{Span}\,S$. </li>
<li class="ltx_item"> It is possible that some vectors are in $\mathrm{Span}\,S$ (such as $\mathbf{w}$), while others are not (such as $\mathbf{y}$). </li>
<li class="ltx_item"> Deciding if a given vector is in $\mathrm{Span}\,S$ leads to a linear system of equations and asking if the system is consistent. </li></ol>
@endcol
@end
@slide
@eg
For the set $S$ defined as in @ref{ABS}, determine if:
\[
\mathbf{v}=\begin{bmatrix}v_{1}\\
\vdots\\
v_{4}\end{bmatrix}\in\mathrm{Span}\,S.
\]
@newcol
Applying Gauss-Jordan elimination to the augmented matrix
\begin{align*}
\displaystyle \left[\begin{array}[]{ccccc|c}1&2&7&1&-1&v_{1}\\
1&1&3&1&0&v_{2}\\
3&2&5&-1&9&v_{3}\\
1&-1&-5&2&0&v_{4}\end{array}\right],
\end{align*}
we obtain

@col
\begin{align*}
\displaystyle \left[\begin{array}[]{ccccc|c}1&0&-1&0&3&-3v_{1}+5v_{2}-v_{4}\\
0&1&4&0&-1&v_{1}-v_{2}\\
0&0&0&1&-2&2v_{1}-3v_{2}+v_{4}\\
0&0&0&0&0&9v_{1}-16v_{2}+v_{3}+4v_{4}\\
\end{array}\right].
\end{align*}
@col
If $9v_{1}-16v_{2}+v_{3}+4v_{4}=0$, then the above matrix is an RREF, and the last column is not a pivot column. In this case, $\mathbf{v}\in\mathrm{Span}\,S$.

@col
If instead $9v_{1}-16v_{2}+v_{3}+4v_{4}\neq 0$, then the linear system corresponding to the augmented matrix is inconsistent,
and thus $\mathbf{v}\not\in\mathrm{Span}\,S$.

@col
We therefore conclude that $\mathbf{v}\in\mathrm{Span}\,S$ if and only if:
\[
9v_{1}-16v_{2}+v_{3}+4v_{4}=0.
\]
@endcol
@end
@slide
@eg
Let
\begin{align*}
\displaystyle R=\{\mathbf{v}_{1},\,\mathbf{v}_{2},\,\mathbf{v}_{3}\}=\left\{\begin{bmatrix}-7\\
5\\
1\end{bmatrix},\,\begin{bmatrix}-6\\
5\\
0\end{bmatrix},\,\begin{bmatrix}-12\\
7\\
4\end{bmatrix}\right\}
\end{align*}
and consider its span $\mathrm{Span}\,R$.
<!--
First, as an example, note that
\begin{align*}
\displaystyle \mathbf{x}=(2)\begin{bmatrix}-7\\
5\\
1\end{bmatrix}+(4)\begin{bmatrix}-6\\
5\\
0\end{bmatrix}+(-3)\begin{bmatrix}-12\\
7\\
4\end{bmatrix}=\begin{bmatrix}-2\\
9\\
-10\end{bmatrix}
\end{align*}
is in $\mathrm{Span}\,R$, since it is a linear combination of $\mathbf{v}_{1},\,\mathbf{v}_{2},\,\mathbf{v}_{3}$. In other words, $\mathbf{x}\in\mathrm{Span}\,R$. Try some different values of $\alpha_{1},\,\alpha_{2},\,\alpha_{3}$ yourself, and see what vectors you can create as elements of $\mathrm{Span}\,R$.
Now ask if a given vector is an element of $\mathrm{Span}\,R$. For example, is -->



@newcol
Does the vector $\mathbf{z}=\begin{bmatrix}-33\\
24\\
5\end{bmatrix}$ lie in $\mathrm{Span}\,R$?

@col
To answer this question, we will look for scalars $\alpha_{1},\,\alpha_{2},\,\alpha_{3}$ so that
\begin{align*}
\displaystyle\alpha_{1}\mathbf{v}_{1}+\alpha_{2}\mathbf{v}_{2}+\alpha_{3}\mathbf{v}_{3}&\displaystyle=\mathbf{z}.
\end{align*}
This is equivalent to finding solutions to the following system of linear equations:
\begin{align*}
\displaystyle-7\alpha_{1}-6\alpha_{2}-12\alpha_{3}&\displaystyle=-33 \\
\displaystyle 5\alpha_{1}+5\alpha_{2}+7\alpha_{3}&\displaystyle=24 \\
\displaystyle\alpha_{1}+4\alpha_{3}&\displaystyle=5.
\end{align*}
Building the augmented matrix for this linear system and row-reducing gives
\begin{align*}
\displaystyle \left[\begin{array}[]{ccc|c}\boxed{1}&0&0&-3\\
0&\boxed{1}&0&5\\
0&0&\boxed{1}&2\end{array}\right].
\end{align*}
This system has a unique solution,
\begin{align*}
\displaystyle\alpha_{1}=-3\quad\displaystyle\alpha_{2}=5\quad\displaystyle\alpha_{3}=2
\end{align*}
telling us that
\begin{align*}
\displaystyle (-3)\mathbf{v}_{1}+(5)\mathbf{v}_{2}+(2)\mathbf{v}_{3}=\mathbf{z}.
\end{align*}
So we are convinced that $\mathbf{z}$ really is in $\mathrm{Span}\,R$. Notice that in this case we have only one way to answer the question affirmatively, since the solution is unique.

@col
Let us ask about another vector.
Let $\mathbf{x}=\begin{bmatrix}-7\\
8\\
-3\end{bmatrix}$.
Is $\mathbf{x}$ a vector in $\mathrm{Span}\,R$?

@col
In other words, are there scalars $\alpha_{1},\,\alpha_{2},\,\alpha_{3}$ so that:
\begin{align*}
\displaystyle\alpha_{1}\mathbf{v}_{1}+\alpha_{2}\mathbf{v}_{2}+\alpha_{3}\mathbf{v}_{3}&\displaystyle=\mathbf{x}
\end{align*}
This is equivalent to finding the solutions to the system of equations

\begin{align*}
\displaystyle-7\alpha_{1}-6\alpha_{2}-12\alpha_{3}&\displaystyle=-7 \\
\displaystyle 5\alpha_{1}+5\alpha_{2}+7\alpha_{3}&\displaystyle=8 \\
\displaystyle\alpha_{1}+4\alpha_{3}&\displaystyle=-3.
\end{align*}
Building the augmented matrix for this linear system and row-reducing gives
\begin{align*}
\displaystyle \left[\begin{array}[]{ccc|c}\boxed{1}&0&0&1\\
0&\boxed{1}&0&2\\
0&0&\boxed{1}&-1\end{array}\right].
\end{align*}
This system has a unique solution,
\begin{align*}
\displaystyle\alpha_{1}=1\quad\displaystyle\alpha_{2}=2\quad\displaystyle\alpha_{3}=-1
\end{align*}
telling us that
\begin{align*}
\displaystyle (1)\mathbf{v}_{1}+(2)\mathbf{v}_{2}+(-1)\mathbf{v}_{3}=\mathbf{x}.
\end{align*}
So we are convinced that $\mathbf{x}$ really is in $\mathrm{Span}\,R$. Notice that in this case we again have only one way to answer the question affirmatively since the solution is again unique.

We could continue to test other vectors for membership in $\mathrm{Span}\,R$, but there is no point. A question about membership in $\mathrm{Span}\,R$ inevitably leads to a system of three equations in the three variables $\alpha_{1},\,\alpha_{2},\,\alpha_{3}$ with a coefficient matrix whose columns are the vectors $\mathbf{v}_{1},\,\mathbf{v}_{2},\,\mathbf{v}_{3}$. This particular coefficient matrix is nonsingular, so by @ref{NME1} the system is guaranteed to have a solution. (This solution is unique, but that is not critical here.) So no matter which vector we might have chosen for $\mathbf{z}$, we are certain to discover that it was an element of $\mathrm{Span}\,R$.

<strong>Conclusion</strong>: Every vector of size 3 is in $\mathrm{Span}\,R$, or $\mathrm{Span}\,R={\mathbb{R}}^{3}$.
@endcol
@end
@slide
The previous example above inspires the following result:
@thm
@label{thm:ANSRM}
Given $m$ vectors $S=\{\mathbf{u}_{1},\,\mathbf{u}_{2},\,\mathbf{u}_{3},\,\ldots,\,\mathbf{u}_{m}\}$ in ${\mathbb{R}}^{m}$, let $A$ be the $m\times m$ square matrix whose $i$-th column is $\mathbf{u}_{i}$. Then $A$ is non-singular if and only if $\mathrm{Span}\,S={\mathbb{R}}^{m}$.
@end
@proof
@newcol
($\Rightarrow$)
@newcol
If $A$ is non-singular, then for every $\mathbf{b}\in{\mathbb{R}}^{m}$ the equation $\mathcal{LS}({A},{\mathbf{b}})$ is consistent
by @ref{b25e99edb5cb21a0655d0d961da21e47}. Hence $\mathbf{b}\in\mathrm{Span}\,S$. So ${\mathbb{R}}^{m}=\mathrm{Span}\,S$.
@endcol

($\Leftarrow$)
@newcol
This part is difficult; we will only sketch the idea.
Let the RREF of $A$ be $B$. Suppose that $A$ is singular. Then $B\neq I_{m}$ and the last row of $B$ is a zero row. So there exists $\mathbf{b}\in{\mathbb{R}}^{m}$ such that $\mathcal{LS}({B},{\mathbf{b}})$ is inconsistent.
(e.g. $\mathbf{b}=\begin{bmatrix}0\\\vdots\\0\\1\end{bmatrix}$).
Hence there exists $\mathbf{c}\in{\mathbb{R}}^{m}$ such that $\mathcal{LS}({A},{\mathbf{c}})$ is inconsistent (why?).
Thus $\mathbf{c}\notin\mathrm{Span}\,S$. So $\mathrm{Span}\,S\neq{\mathbb{R}}^{m}$. This completes the proof.
<hr/> Alternatively: 
@newcol
Suppose $A$ is singular. We claim that the span of $S$ is not equal to $\mathbb{R}^m$:

@col
If $A$ is singular, then there exists a sequence of elementary matrices $J_i$ such that:
$J_lJ_{l - 1}\cdots J_2J_1 A$ is an RREF matrix $B$ whose last row is a zero row.

@col
Let $J = J_lJ_{l - 1}\cdots J_2J_1$, which is invertible.
We claim that the vector:
\[
\vec{v} = J^{-1}\vec{e}_m = J^{-1}\begin{bmatrix}0\\0\\\vdots\\0\\1\end{bmatrix}
\]
(which incidentally is the last column of the matrix $J^{-1}$)
does not lie in $\mathrm{Span}\,S$.

@col
Suppose $\vec{v} \in \mathrm{Span}\,S$, then exists a vector $\vec{x} \in \mathbb{R}^m$ such that:
\[
A\vec{x} = \vec{v} = J^{-1}\vec{e}_m
\]
@col
Multiplying both sides with $J$ from the left, we have:
\[
JA\vec{x} = \vec{e}_m.
\]
But $JA = B$, whose last row is the zero row, which implies that the last component of the vector $JA\vec{x}$
is equal to zero, contradicting the fact that the last component of $\vec{e}_m$ is equal to $1$.

@col
It follows that if $A$ is singular, then $\mathrm{Span}\,S \neq \mathbb{R}^m$.
@endcol
@qed
@endcol
@endcol
@end
@section{Obtain Same Span Using Fewer Vectors}
@eg
Begin with the following set of four vectors of size $3$:
\begin{align*}
\displaystyle T
=\left\{\mathbf{w}_{1},\,\mathbf{w}_{2},\,\mathbf{w}_{3},\,\mathbf{w}_{4}\right\}
=\left\{\begin{bmatrix}2\\
-3\\
1\end{bmatrix},\,\begin{bmatrix}1\\
4\\
1\end{bmatrix},\,\begin{bmatrix}7\\
-5\\
4\end{bmatrix},\,\begin{bmatrix}-7\\
-6\\
-5\end{bmatrix}
\right\}.
\end{align*}
@newcol
Let:
\begin{align*}
\displaystyle D=\begin{bmatrix}2&1&7&-7\\
-3&4&-5&-6\\
1&1&4&-5\end{bmatrix}
\end{align*}
and consider the infinite set $W=\mathrm{Span}\,T$. Check that the vector
\begin{align*}
\displaystyle \mathbf{z}_{2}=\begin{bmatrix}2\\
3\\
0\\
1\end{bmatrix}
\end{align*}
is a solution to the homogeneous system $\mathcal{LS}({D},{\mathbf{0}})$

@col
We can write the linear combination,
\begin{align*}
\displaystyle 2\mathbf{w}_{1}+3\mathbf{w}_{2}+0\mathbf{w}_{3}+1\mathbf{w}_{4}=\mathbf{0}
\end{align*}
which we can solve for $\mathbf{w}_{4}$ as
\begin{align*}
\displaystyle \mathbf{w}_{4}=(-2)\mathbf{w}_{1}+(-3)\mathbf{w}_{2}.
\end{align*}
@col
This equation says that whenever we encounter the vector $\mathbf{w}_{4}$, we can replace it with a specific linear combination of the vectors $\mathbf{w}_{1}$ and $\mathbf{w}_{2}$. So using $\mathbf{w}_{4}$ in the set $T$, along with $\mathbf{w}_{1}$ and $\mathbf{w}_{2}$, is excessive. An example of what we mean here can be illustrated by the computation:

@col
\begin{align*}
\displaystyle 5\mathbf{w}_{1}&\displaystyle+(-4)\mathbf{w}_{2}+6\mathbf{w}_{3}+(-3)\mathbf{w}_{4} \\
&\displaystyle=5\mathbf{w}_{1}+(-4)\mathbf{w}_{2}+6\mathbf{w}_{3}+(-3)\left((-2)\mathbf{w}_{1}+(-3)\mathbf{w}_{2}\right) \\
&\displaystyle=5\mathbf{w}_{1}+(-4)\mathbf{w}_{2}+6\mathbf{w}_{3}+\left(6\mathbf{w}_{1}+9\mathbf{w}_{2}\right) \\
&\displaystyle=11\mathbf{w}_{1}+5\mathbf{w}_{2}+6\mathbf{w}_{3}.
\end{align*}
@col
So, what began as a linear combination of the vectors $\mathbf{w}_{1},\,\mathbf{w}_{2},\,\mathbf{w}_{3},\,\mathbf{w}_{4}$ has been reduced to a linear combination of the vectors $\mathbf{w}_{1},\,\mathbf{w}_{2},\,\mathbf{w}_{3}$.

Hence:
@col
\begin{align*}
\displaystyle W=\mathrm{Span}\left\{\mathbf{w}_{1},\,\mathbf{w}_{2},\,\mathbf{w}_{3}\right\},
\end{align*}
and the span of our set of vectors, $W$, has not changed, but we have described it by the span of a set of three vectors, rather than four. Furthermore, we can achieve yet another, similar, reduction.

@col
Check that the vector
\begin{align*}
\displaystyle \mathbf{z}_{1}=\begin{bmatrix}-3\\
-1\\
1\\
0\end{bmatrix}
\end{align*}
is a solution to the homogeneous system $\mathcal{LS}({D},{\mathbf{0}})$.

@col
We can write the linear combination,
\begin{align*}
\displaystyle (-3)\mathbf{w}_{1}+(-1)\mathbf{w}_{2}+1\mathbf{w}_{3}=\mathbf{0}
\end{align*}
which we can solve for $\mathbf{w}_{3}$ as
\begin{align*}
\displaystyle \mathbf{w}_{3}=3\mathbf{w}_{1}+1\mathbf{w}_{2}.
\end{align*}
@col
This equation says that whenever we encounter the vector $\mathbf{w}_{3}$, we can replace it with a specific linear combination of the vectors $\mathbf{w}_{1}$ and $\mathbf{w}_{2}$. So, as before, the vector $\mathbf{w}_{3}$ is not needed in the description of $W$, provided we have $\mathbf{w}_{1}$ and $\mathbf{w}_{2}$ available. In particular, a careful proof would show that
\begin{align*}
\displaystyle W=\mathrm{Span}\left\{\mathbf{w}_{1},\,\mathbf{w}_{2}\right\}
\end{align*}
@col
So $W$ began life as the span of a set of four vectors. We have now shown (utilizing solutions to a homogeneous system) that $W$ can also be described as the span of a set of just two vectors. Convince yourself that we cannot go any further. In other words, it is not possible to dismiss either $\mathbf{w}_{1}$ or $\mathbf{w}_{2}$ in a similar fashion and winnow the set down to just one vector.

@col
What was it about the original set of four vectors that allowed us to declare certain vectors as surplus? And just which vectors were we able to dismiss? And why did we have to stop once we had two vectors remaining? The answers to these questions motivate @keyword{linear independence}, our next section and next definition, and so are worth considering carefully now.
@endcol
@end
@slide
In general:
@thm
@label{thm:spanredundancy}
Let $\{\vect{v}_1, \vect{v}_2, \ldots, \vect{v}_n\}$ be a set of vectors in a vector space.

If $\vect{w}$ lies in $\mathrm{Span}\,\{\vect{v}_1, \vect{v}_2, \ldots, \vect{v}_n\}$, then:
\[
\mathrm{Span}\{\vect{v}_1, \vect{v}_2, \ldots, \vect{v}_n,\vect{w}\}
=
\mathrm{Span}\{\vect{v}_1, \vect{v}_2, \ldots, \vect{v}_n\}
\]
@end
@section{Geometric Interpretation}
<iframe width="800" height="600" allowfullscreen="" style="border: 1px solid rgb(228, 228, 228); border-radius: 4px; overflow: hidden; background: none;" frameborder="0" data-src="https://www.geogebra.org/3d/tnnuszqm?embed" rendered="0" id="iFrameResizer5" scrolling="no" src="https://www.geogebra.org/3d/tnnuszqm?embed"/>
@section{Spanning Sets of Null Spaces}
Recall @ref{thm:nullspacesubspace}
<!-- The following theorem describe how to express a null space as $\mathrm{Span}\,S$. -->



@thm
@label{SSNS}
@newcol
Spanning
Sets for Null Spaces
Suppose that $A$ is an $m\times n$ matrix and $B$ is a row-equivalent matrix in reduced row-echelon form. Suppose that $B$ has $r$ pivot columns, with indices given by $D=\{d_{1},\,d_{2},\,d_{3},\,\ldots,\,d_{r}\}$, while the $n-r$ non-pivot columns have indices $F=\{f_{1},\,f_{2},\,f_{3},\,\ldots,\,f_{n-r}\}$. Construct the $n-r$ vectors $\mathbf{z}_{j}$, $1\leq j\leq n-r$ of size $n$,
\begin{align*}
\displaystyle \left[\mathbf{z}_{j}\right]_{i}=\begin{cases}1&\text{if $i\in F$, $i=f_{j}$}\\
0&\text{if $i\in F$, $i\neq f_{j}$}\\
-\left[B\right]_{k,f_{j}}&\text{if $i\in D$, $i=d_{k}$}\end{cases}
\end{align*}
Then the null space of $A$ is given by
\begin{align*}
\displaystyle {\mathcal{N}}\!\left(A\right)=\mathrm{Span}\left\{\mathbf{z}_{1},\,\mathbf{z}_{2},\,\mathbf{z}_{3},\,\ldots,\,\mathbf{z}_{n-r}\right\}
\end{align*}
@endcol
@end
@proof
@newcol
The can be seen by moving the free variables to another side. For details. See Beezer p88. Don’t memorize this theorem. Instead, study the examples below.
@qed
@endcol
@end
@slide
@eg
<strong>Spanning set of a null space</strong>

Find a set of vectors, $S$, so that the null space of the matrix
\begin{align*}
\displaystyle A=\begin{bmatrix}1&3&3&-1&-5\\
2&5&7&1&1\\
1&1&5&1&5\\
-1&-4&-2&0&4\end{bmatrix}
\end{align*}
is the span of $S$, that is, $\mathrm{Span}\,S={\mathcal{N}}\!\left(A\right)$.
<hr/>
@newcol
The null space of $A$ is the set of all solutions to the homogeneous system $\mathcal{LS}({A},{\mathbf{0}})$.
Begin by row-reducing $A$. The result is
\begin{align*}
\displaystyle \begin{bmatrix}\boxed{1}&0&6&0&4\\
0&\boxed{1}&-1&0&-2\\
0&0&0&\boxed{1}&3\\
0&0&0&0&0\end{bmatrix}.
\end{align*}
We have $D=\{1,\,2,\,4\}$ and $F=\{3,\,5\}$. Hence $x_{3}$ and $x_{5}$ are free variables and we can interpret each nonzero row as an expression for the dependent variables $x_{1}$, $x_{2}$, $x_{4}$ (respectively) in the free variables $x_{3}$ and $x_{5}$. With this we can write the vector form of a solution vector as
\begin{align*}
\displaystyle \begin{bmatrix}x_{1}\\
x_{2}\\
x_{3}\\
x_{4}\\
x_{5}\end{bmatrix}=\begin{bmatrix}-6x_{3}-4x_{5}\\
x_{3}+2x_{5}\\
x_{3}\\
-3x_{5}\\
x_{5}\end{bmatrix}=x_{3}\begin{bmatrix}-6\\
1\\
1\\
0\\
0\end{bmatrix}+x_{5}\begin{bmatrix}-4\\
2\\
0\\
-3\\
1\end{bmatrix}.
\end{align*}
Then, in the notation of the above theorem, we have
\begin{align*}
\displaystyle\mathbf{z}_{1}&\displaystyle=\begin{bmatrix}-6\\
1\\
1\\
0\\
0\end{bmatrix}&\displaystyle\mathbf{z}_{2}&\displaystyle=\begin{bmatrix}-4\\
2\\
0\\
-3\\
1\end{bmatrix}
\end{align*}
and
\begin{align*}
\displaystyle {\mathcal{N}}\!\left(A\right)
=\mathrm{Span}\,\left\{ \mathbf{z}_{1},\,\mathbf{z}_{2}
\right\}
=\mathrm{Span}
\left
\{\begin{bmatrix}-6\\
1\\
1\\
0\\
0\end{bmatrix},\,\begin{bmatrix}-4\\
2\\
0\\
-3\\
1\end{bmatrix}
\right\}.
\end{align*}
@endcol
@end
@eg
@newcol
Consider the matrix:
\begin{align*}
\displaystyle A=\begin{bmatrix}2&1&5&1&5&1\\
1&1&3&1&6&-1\\
-1&1&-1&0&4&-3\\
-3&2&-4&-4&-7&0\\
3&-1&5&2&2&3\end{bmatrix}.
\end{align*}
Row-reducing $A$ gives the matrix
\begin{align*}
\displaystyle B=\begin{bmatrix}\boxed{1}&0&2&0&-1&2\\
0&\boxed{1}&1&0&3&-1\\
0&0&0&\boxed{1}&4&-2\\
0&0&0&0&0&0\\
0&0&0&0&0&0\end{bmatrix}.
\end{align*}
First, the non-pivot columns have indices $F=\{3,\,5,\,6\}$, so we will construct the $n-r=6-3=3$ vectors with a pattern of zeros and ones dictated by the indices in $F$. This is the realization of the first two lines of the three-case definition of the vectors $\mathbf{z}_{j}$, $1\leq j\leq n-r$.
\begin{align*}
\displaystyle\mathbf{z}_{1}&\displaystyle=\begin{bmatrix}\\
\\
1\\
\\
0\\
0\end{bmatrix}&\displaystyle\mathbf{z}_{2}&\displaystyle=\begin{bmatrix}\\
\\
0\\
\\
1\\
0\end{bmatrix}&\displaystyle\mathbf{z}_{3}&\displaystyle=\begin{bmatrix}\\
\\
0\\
\\
0\\
1\end{bmatrix}.
\end{align*}
Each of these vectors arises due to the presence of a non-pivot column. The remaining entries of each vector are the entries of the non-pivot column, negated, and distributed into the empty slots in order (these slots have indices in the set $D$, so also refer to pivot columns). This is the realization of the third line of the three-case definition of the vectors $\mathbf{z}_{j}$, $1\leq j\leq n-r$.
\begin{align*}
\displaystyle\mathbf{z}_{1}&\displaystyle=\begin{bmatrix}-2\\
-1\\
1\\
0\\
0\\
0\end{bmatrix}&\displaystyle\mathbf{z}_{2}&\displaystyle=\begin{bmatrix}1\\
-3\\
0\\
-4\\
1\\
0\end{bmatrix}&\displaystyle\mathbf{z}_{3}&\displaystyle=\begin{bmatrix}-2\\
1\\
0\\
2\\
0\\
1\end{bmatrix}.
\end{align*}
So we have
\begin{align*}
\displaystyle {\mathcal{N}}\!\left(A\right)=\mathrm{Span}\,\left\{ \mathbf{z}_{1},\,\mathbf{z}_{2},\,\mathbf{z}_{3}
\right\}
=
\mathrm{Span}\,\left\{ \begin{bmatrix}-2\\
-1\\
1\\
0\\
0\\
0\end{bmatrix},\,\begin{bmatrix}1\\
-3\\
0\\
-4\\
1\\
0\end{bmatrix},\,\begin{bmatrix}-2\\
1\\
0\\
2\\
0\\
1\end{bmatrix}
\right\}.
\end{align*}
@endcol
@end