In this section we’re going to take a look at an important
idea in the study of vector spaces. We
will also be drawing heavily on the ideas from the previous two sections and so
make sure that you are comfortable with the ideas of span and linear
independence.
We’ll start this section off with the following definition.
|
Definition 1 Suppose
 is a set of vectors from the vector space V.
Then S is called a basis (plural is bases) for V if both of
the following conditions hold.
(a) ,
i.e. S spans the vector space V.
(b) S is a linearly independent set of
vectors.
|
Let’s take a look at some examples.
|
Example 1 Determine
if each of the sets of vectors will be a basis for .
(a) ,
and . [Solution]
(b) ,
and . [Solution]
(c) and . [Solution]
(d) ,
and . [Solution]
Solution
(a) ,
and .
Now, let’s see what we’ve got to do here to determine
whether or not this set of vectors will be a basis for . First, we’ll need to show that these
vectors span and from the section on Span
we know that to do this we need to determine if we can find scalars ,
,
and so that a general vector from can be expressed as a linear combination of
these three vectors or,
As we saw in
the section on Span all we need to do is convert this to a system of
equations, in matrix form, and then determine if the coefficient matrix has a
non-zero determinant or not. If the
determinant of the coefficient matrix is non-zero then the set will span the
given vector space and if the determinant of the coefficient matrix is zero
then it will not span the given vector space.
Recall as well that if the determinant of the coefficient matrix is
non-zero then there will be exactly one solution to this system for each u.
The matrix form
of the system is,
Before we get
the determinant of the coefficient matrix let’s also take a look at the other
condition that must be met in order for this set to be a basis for . In order for these vectors to be a basis
for then they must be linearly independent. From the section on Linear Independence we know that to
determine this we need to solve the following equation,
If this system
has only the trivial solution the vectors will be linearly independent and if
it has solutions other than the trivial solution then the vectors will be
linearly dependent.
Note however,
that this is really just a specific case of the system that we need to solve
for the span question. Namely here we
need to solve,
Also, as noted
above, if these vectors will span then there will be exactly one solution to
the system for each u. In this case we know that the trivial
solution will be a solution, our only question is whether or not it is the
only solution.
So, all that we
need to do here is compute the determinant of the coefficient matrix and if
it is non-zero then the vectors will both span and be linearly independent and hence the
vectors will be a basis for . On the other hand, if the determinant is
zero then the vectors will not span and will not be linearly independent and so
they won’t be a basis for .
So, here is the
determinant of the coefficient matrix for this problem.
So, these
vectors will form a basis for .
[Return to Problems]
(b) ,
and .
Now, we could
use a similar path for this one as we did earlier. However, in this case, we’ve done all the
work for this one in previous sections.
In Example 4(a) of the section on
Span we determined that the standard basis vectors (Interesting name isn’t
it? We’ll come back to this in a bit) ,
and will span . Notice that while we’ve changed the
notation a little just for this problem we are working with the standard
basis vectors here and so we know that they will span .
Likewise, in Example 1(c) from the section on
Linear Independence we saw that these vectors are linearly independent.
Hence based on
all this previous work we know that these three vectors will form a basis for
.
[Return to Problems]
(c) and .
We can’t use
the method from part (a) here because the coefficient matrix wouldn’t be
square and so we can’t take the determinant of it. So, let’s just start this out by checking
to see if these two vectors will span . If these two vectors will span then for each in there must be scalars and so that,
However, we can
see right away that there will be problems here. The third component of the each of these
vectors is zero and hence the linear combination will never have any non-zero
third component. Therefore, if we
choose to be any vector in with we will not be able to find scalars and to satisfy the equation above.
Therefore,
these two vectors do not span and hence cannot be a basis for .
Note however,
that these two vectors are linearly independent (you should verify
that). Despite this however, the
vectors are still not a basis for since they do not span .
(d) ,
and
[Return to Problems]
In this case
we’ve got three vectors with three components and so we can use the same
method that we did in the first part.
The general equation that needs solved here is,
and the matrix
form of this is,
We’ll leave it to you to verify that and so these three vectors do not span and are not linearly independent. Either of which will mean that these three
vectors are not a basis for .
[Return to Problems]
|
Before we move on let’s go back and address something we
pointed out in Example 1(b). As we
pointed out at the time the three vectors we were looking at were the standard
basis vectors for . We should discuss the name a little more at
this point and we’ll do it a little more generally than in .
The vectors
will span as we saw in the section on Span and it is fairly
simple to show that these vectors are linearly independent (you should verify
this) and so they form a basis for . In some way this set of vectors is the
simplest (we’ll see this in a bit) and so we call them the standard basis vectors for .
We also have a set of standard basis vectors for a couple of
the other vector spaces we’ve been looking at occasionally. Let’s take a look at each of them.
Note that we only looked at the standard basis vectors for ,
but you should be able to modify this appropriately to arrive at a set of
standard basis vector for in general.
Next let’s take a look at the following theorem that gives
us one of the reasons for being interested in a set of basis vectors.
|
Theorem 1 Suppose
that the set is a basis for the vector space V then every vector u from V can be expressed as a linear combination of the vectors from S in exactly one way.
|
Proof : First,
since we know that the vectors in S
are a basis for V then for any vector
u in V we can write it as a linear combination as follows,
Now, let’s suppose that it is also possible to write it as
the following linear combination,
If we take the difference of these two linear combinations
we get,
However, because the vectors in S are a basis they are linearly independent. That means that this equation can only have
the trivial solution. Or, in other words,
we must have,
But this means that,
and so the two linear combinations were in fact the same
linear combination.
We also have the following fact. It probably doesn’t really rise to the level
of a theorem, but we’ll call it that anyway.
The proof here is so simple that we’re not really going to
give it. By assumption the set is
linearly independent and by definition V
is the span of S and so the set must
be a basis for V.
We now need to take a look at the following definition.
|
Definition 2 Suppose
that V is a non-zero vector space
and that S is a set of vectors from
V that form a basis for V.
If S contains a finite
number of vectors, say ,
then we call V a finite dimensional vector space and
we say that the dimension of V, denoted by ,
is n (i.e. the number of basis elements in S). If V is not a finite dimensional vector space (so S does not have a finite number of
vectors) then we call it an infinite
dimensional vector space.
By definition the dimension of the zero vector space (i.e. the vector space consisting
solely of the zero vector) is zero.
|
Here are the dimensions of some of the vector spaces we’ve
been dealing with to this point.
We now need to take a look at several important theorems
about vector spaces. The first couple of
theorems will give us some nice ideas about linearly independent/dependent sets
and spans. One of the more important
uses of these two theorems is constructing a set of basis vectors as we’ll see
eventually.
|
Theorem 3 Suppose
that V is a vector space and that is any basis for V.
(a) If
a set has more than n vectors then
it is linearly dependent.
(b) If
a set has fewer than n vectors then
it does not span V.
|
Proof :
(a) Let and suppose that . Since S
is a basis of V every vector in R can be written as a linear combination
of vectors from S as follows,
Now, we want to show that the vectors in R are linearly dependent. So, we’ll need to show that there are more
solutions than just the trivial solution to the following equation.
If we plug in the set of linear combinations above for the ’s in this equation and collect all the
coefficients of the ’s we arrive at.
Now, the ’s are linearly independent and so we
know that the coefficients of each of the in this equation must be zero. This gives the following system of equations.
Now, in this system the ’s are known scalars from the linear
combinations above and the ’s are unknowns. So we can see that there are n equations and m unknowns. However, because
there are more unknowns than equations and so
by Theorem 2 in the
solving systems of equations section we know that if there are more unknowns
than equations in a homogeneous system, as we have here, there will be
infinitely many solutions.
Therefore the equation,
will have more solutions than the trivial solution and so
the vectors in R must be linearly
dependent.
(b) The proof of
this part is very similar to the previous part.
Let’s start with the set and this time we’re going to assume that . It’s not so easy to show directly that R will not span V, but if we assume for a second that R does span V we’ll see
that we’ll run into some problems with our basis set S. This is called a proof by
contradiction. We’ll assume the opposite
of what we want to prove and show that this will lead to a contradiction of
something that we know is true (in this case that S is a basis for V).
So, we’ll assume that R
will span V. This means that all the vectors in S can be written as a linear combination
of the vectors in R or,
Let’s now look at the equation,
Now, because S is
a basis we know that the ’s must be linearly independent and so
the only solution to this must be the trivial solution. However, if we substitute the linear
combinations of the ’s into this, rearrange as we did in
part (a) and then setting all the
coefficients equal to zero gives the following system of equations.
Again, there are more unknowns than equations here and so
there are infinitely many solutions.
This contradicts the fact that we know the only solution to the equation
is the trivial solution.
So, our original assumption that R spans V must be
wrong. Therefore R will not span V.
Proof :
(a) If we need to show that the set is linearly independent. So, let’s form the equation,
Now, if is not zero we will be able to write u as a linear combination of the ’s but this contradicts the fact that u is not in . Therefore we must have and our equation is now,
But the vectors in S
are linearly independent and so the only solution to this is the trivial
solution,
So, we’ve shown that the only solution to
is
Therefore, the vectors in R are linearly independent.
(b) Let’s suppose
that our set is and so we have . First, by assumption u is a linear combination of the remaining vectors in S or,
Next let w be any
vector in . So, w
can be written as a linear combination of all the vectors in S or,
Now plug in the expression for u above to get,
So, w is a linear
combination of vectors only in R and
so at the least every vector that is in must also be in .
Finally, if w is
any vector in then it can be written as a linear combination
of vectors from R, but since these
are also vectors in S we see that w can also, by default, be written as a
linear combination of vectors from S
and so is also in . We’ve just shown that every vector in must also be in .
Since we’ve shown that must be contained in and that every vector in must also be contained in this can only be true if .
We can use the previous two theorems to get some nice ideas
about the basis of a vector space.
|
Theorem 5 Suppose
that V is a vector space then all
the bases for V contain the same
number of vectors.
|
Proof : Suppose
that is a basis for V. Now, let R be any other basis for V.
Then by Theorem 3 above if R
contains more than n elements it
can’t be a linearly independent set and so can’t be a basis. So, we know that, at the least R can’t contain more than n elements. However, Theorem 3 also tells us that if R contains less than n elements then it won’t span V and hence can’t be a basis for V.
Therefore the only possibility is that R must contain exactly n
elements.
Proof : First
suppose that . If S
is linearly dependent then there must be some vector u in S that can be
written as a linear combination of other vectors in S and so by Theorem 4(b) we can remove u from S and our new set
of vectors will still span V. However, Theorem 3(b)
tells us that any set with fewer vectors than a basis (i.e. less than n in this
case) can’t span V. Therefore, S must be linearly independent and hence S is a basis for V.
Now, let’s suppose that S
is linearly independent. If S does not span V then there must be a vector u
that is not in . If we add u to S the resulting set
with vectors must be linearly independent by
Theorem 4(a). On the other hand, Theorem
3(a) tells us that any set with more vectors than the basis (i.e. greater than n) can’t be linearly independent.
Therefore, S must span V and hence S is a basis for V.
|
Theorem 7 Suppose
that V is a finite dimensional
vector space with and that S
is any finite set of vectors from V.
(a) If
S spans V but is not a basis for V
then it can be reduced to a basis for V
by removing certain vectors from S.
(b) If
S is linearly independent but is
not a basis for V then it can be
enlarged to a basis for V by adding
in certain vectors from V.
|
Proof :
(a) If S spans V but is not a basis for V
then it must be a linearly dependent set.
So, there is some vector u in
S that can be written as a linear
combination of the other vectors in S. Let R
be the set that results from removing u
from S. Then by Theorem 4(b) R will still span V. If R
is linearly independent then we have a basis for V and if it is still linearly dependent we can remove another
element to form a new set that will still span V. We continue in this way
until we’ve reduced S down to a set
of linearly independent vectors and at that point we will have a basis of V.
(b) If S is linearly independent but not a
basis then it must not span V. Therefore, there is a vector u that is not in . So, add u
to S to form the new set R.
Then by Theorem 4(a) the set R
is still linearly independent. If R now spans V we’ve got a basis for V
and if not add another element to form the new linearly independent set . Continue in this fashion until we reach a set
with n vectors and then by Theorem 6
this set must be a basis for V.
Okay we should probably see some examples of some of these
theorems in action.
|
Example 5 Reduce
each of the following sets of vectors to obtain a basis for the given vector
space.
(a) ,
,
and for . [Solution]
(b) ,
,
,
and for . [Solution]
Solution
First, notice that provided each of these sets of vectors
spans the given vector space Theorem 7(a) tells us that this can in fact be
done.
(a) ,
,
and for .
We will leave it to you to verify that this set of vectors
does indeed span and since we know that we can see that we’ll need to remove one
vector from the list in order to get down to a basis. However, we can’t just remove any of the
vectors. For instance if we removed the set would no longer span . You should verify this, but you can also
quickly see that only has a non-zero first component and so will
be required for the vectors to span .
Theorem 4(b) tells us that if we remove a vector that is a
linear combination of some of the other vectors we won’t change the span of
the set. So, that is what we need to
look for. Now, it looks like the last
three vectors are probably linearly dependent so if we set up the following
equation
and solve it,
we can see that
these in fact are linearly dependent vectors.
This means that we can remove any of these since we could write any
one of them as a linear combination of the other two. So, let’s remove for no other reason that the entries in this
vector are larger than the others.
The following
set still spans and has exactly 3 vectors and so by Theorem
6 it must be a basis for .
For the
practice you should verify that this set does span and is linearly independent.
[Return to Problems]
(b) ,
,
,
and for .
We’ll go through this one a little faster. First, you should verify that the set of
vectors does indeed span . Also, because we know that we’ll need to remove two of the
vectors. Again, remember that each
vector we remove must be a linear combination of some of the other vectors.
First, it looks like is a linear combination of and (you should verify this) and so we can
remove and the set will still span . This leaves us with the following set of
vectors.
Now, it looks
like can easily be written as a linear
combination of the remaining vectors (again, please verify this) and so we
can remove that one as well.
We now have the
following set,
which has 3 vectors and will span and so it must be a basis for by Theorem 6.
[Return to Problems]
|
|
Example 6 Expand
each of the following sets of vectors into a basis for the given vector
space.
(a) ,
,
in . [Solution]
(b) and in . [Solution]
Solution
Theorem 7(b) tells us that this is possible to do
provided the sets are linearly
independent.
(a) ,
,
in .
We’ll leave it to you to verify that these vectors are
linearly independent. Also, and so it looks like we’ll just need to add
in a single vector to get a basis.
Theorem 4(a) tells us that provided the vector we add in is not in the
span of the original vectors we can retain the linear independence of the
vectors. This will in turn give us a
set of 4 linearly independent vectors and so by Theorem 6 will have to be a
basis for .
Now, we need to find a vector that is not in the span of
the given vectors. This is easy to do
provided you notice that all of the vectors have a zero in the fourth
component. This means that all the
vectors that are in will have a zero in the fourth
component. Therefore, all that we need
to do is take any vector that has a non-zero fourth component and we’ll have
a vector that is outside . Here are some possible vectors we could
use,
The last one
seems to be in keeping with the pattern of the original three vectors so
we’ll use that one to get the following set of four vectors.
Since this set
is still linearly independent and now has 4 vectors by Theorem 6 this set is
a basis for (you should verify this).
[Return to Problems]
(b) and in .
The two vectors
here are linearly independent (verify this) and and so we’ll need to add in two vectors to
get a basis. We will have to do this
in two steps however. The first vector
we add cannot be in and the second vector we add cannot be in where is the new vector we added in the first
step.
So, first notice that all the vectors in will have zeroes in the second column so
anything that doesn’t have a zero in at least one entry in the second column
will work for . We’ll choose the following for .
Note that this
is probably not the best choice since it’s got non-zero entries in both
entries of the second column. It would
have been easier to choose something that had a zero in one of the entries of
the second column. However, if we
don’t do that this will allow us make a point about choosing the second
vector. Here is the list of vectors
that we’ve got to this point.
Now, we need to
find a fourth vector and it needs to be outside of . Now, let’s again note that because of our
choice of all the vectors in will have identical numbers in both entries
of the second column and so we can chose any new vector that does not have
identical entries in the second column and we’ll have something that is
outside of . Again, we’ll go with something that is
probably not the best choice if we had to work with this basis, but let’s not
get too locked into always taking the easy choice. There are, on occasion, reasons to choose
vectors other than the “obvious” and easy choices. In this case we’ll use,
This gives us
the following set of vectors,
and they will be a basis for since these are four linearly independent
vectors in a vector space with dimension of 4.
[Return to Problems]
|
We’ll close out this section with a couple of theorems and
an example that will relate the dimensions of subspaces of a vector space to
the dimension of the vector space itself.
|
Theorem 8 Suppose
that W is a subspace of a finite
dimensional vector space V then W is also finite dimensional.
|
Proof : Suppose
that . Let’s also suppose that W is not finite dimensional and suppose that S is a basis for W. Since we’ve assumed that W is not finite dimensional we know that S will not have a finite number of vectors in it. However, since S is a basis for W we
know that they must be linearly independent and we also know that they must be
vectors in V. This however, means that we’ve got a set of
more than n vectors that is linearly
independent and this contradicts the results of Theorem 3(a).
Therefore W must
be finite dimensional as well.
We can actually go a step further here than this theorem.
Proof : By
Theorem 8 we know that W must be a
finite dimensional vector space and so let’s suppose that is a basis for W. Now, S is either a basis for V
or it isn’t a basis for V.
If S is a basis
for V then by Theorem 5 we have that .
On the other hand, if S
is not a basis for V by Theorem 7(b)
(the vectors of S must be linearly
independent since they form a basis for W)
it can be expanded into a basis for V
and so we then know that .
So, we’ve shown that in every case we must have .
Now, let’s just assume that all we know is that . In this case S will be a set of n
linearly independent vectors in a vector space of dimension n (since ) and so by Theorem 6, S must be a basis for V
as well. This means that any vector u from V can be written as a linear combination of vectors from S.
However, since S is also a
basis for W this means that u must also be in W.
So, we’ve just shown that every vector in V must also be in W, and because W is a
subspace of V we know that every
vector in W is also in V.
The only way for this to be true is if we have .
We should probably work one quick example illustrating this theorem.
|
Example 7 Determine
a basis and dimension for the null
space of
Solution
First recall that to find the null space of a matrix we
need to solve the following system of equations,
We solved a
similar system back in Example
7 of the Solving Systems of Equation section so we’ll leave it to you to
verify that the solution is,
Now, recall that the null space of an matrix will be a subspace of so the null space of this matrix must be a
subspace of and so its dimension should be 5 or less.
To verify this
we’ll need the basis for the null space.
This is actually easier to find than you might think. The null space will consist of all vectors
in that have the form,
Now, split this
up into two vectors. One that contains
only terms with a t in them and one
that contains only term with an s
in them. Then factor the t and s out of the vectors.
So, we can see
that the null space is the space that is the set of all vectors that are a
linear combination of
and so the null space of A is spanned by these two vectors. You should also verify that these two
vectors are linearly independent and so they in fact form a basis for the
null space of A. This also means that the null space of A has a dimension of 2 which is less
than 5 as Theorem 9 suggests it should be.
|