You can navigate through this E-Book using the menu to the left. For E-Books that have a Chapter/Section organization each option in the menu to the left indicates a chapter and will open a menu showing the sections in that chapter. Alternatively, you can navigate to the next/previous section or chapter by clicking the links in the boxes at the very top and bottom of the material.
Also, depending upon the E-Book, it will be possible to download the complete E-Book, the chapter containing the current section and/or the current section. You can do this be clicking on the E-Book, Chapter, and/or the Section link provided below.
For those pages with mathematics on them you can, in most cases, enlarge the mathematics portion by clicking on the equation. Click the enlarged version to hide it.
In this section we want to take a look at some important
subspaces that are associated with matrices.
In fact they are so important that they are often called the fundamental
subspaces of a matrix. We’ve actually
already seen one of the fundamental subspaces, the null space, previously although we will give its
definition here again for the sake of completeness.
Before we give the formal definitions of the fundamental
subspaces we need to quickly review a concept that we first saw back when we
were looking at matrix arithmetic.
Given an 
matrix
The row vectors
(we called them row matrices at the time) are the vectors in formed out of the rows of A. The column vectors (again we called them column matrices at the time)
are the vectors in that are formed out of the columns of A.
|
Example 1 Write
down the row vectors and column vectors for
Solution
The row vectors
are,
The column
vectors are
|
Note that despite the fact that we’re calling them vectors
we are using matrix notation for them.
The reason is twofold. First,
they really are row/column matrices and so we may as well denote them as such
and second in this way we can keep the “orientation” of each to remind us
whether or not they are row vectors or column vectors. In other words, row vectors are listed
horizontally and column vectors are listed vertically.
Because we’ll be using the matrix notation for the row and
column vectors we’ll be using matrix notation for vectors in general in this
section so we won’t be mixing and matching the notations too much.
Here then are the definitions of the three fundamental
subspaces that we’ll be investigating in this section.
We are going to be particularly interested in the basis for
each of these subspaces and that in turn means that we’re going to be able to
discuss the dimension of each of them.
At this point we can give the notation for the dimension of the null
space, but we’ll need to wait a bit before we do so for the row and column spaces. The reason for the delay will be apparent
once we reach that point. So, let’s go
ahead and give the notation for the null space.
|
Definition 2 The
dimension of the null space of A is
called the nullity of A and is denoted by .
|
We should work an example at this point. Because we’ve already seen how to find the
basis for the null space (Example 4(b)
in the Subspaces section and Example 7 of
the Basis section) we’ll do one example at this point and then devote the
remainder of the discussion on basis/dimension of these subspaces to finding
the basis/dimension for the row and column space. Note that we will see an example or two later
in this section of null spaces as well.
|
Example 2 Determine
a basis for the null space of the following matrix.
Solution
So, to find the
null space we need to solve the following system of equations.
We’ll leave it
to you to verify that the solution is given by,
In matrix form
the solution can be written as,
So, the
solution can be written as a linear combination of the three linearly
independent vectors (verify the linearly independent claim!)
and so these
three vectors then form the basis for the null space since they span the null
space and are linearly independent.
Note that this also means that the null space has a dimension of 3
since there are three basis vectors for the null space and so we can see that
|
Again, remember that we’ll be using matrix notation for
vectors in this section.
Okay, now that we’ve gotten an example of the basis for the
null space taken care of we need to move onto finding bases (and hence the
dimensions) for the row and column spaces of a matrix. However, before we do that we first need a
couple of theorems out of the way. The
first theorem tells us how to find the basis for a matrix that is in row-echelon
form.
|
Theorem 1 Suppose
that the matrix U is in row-echelon
form. The row vectors containing
leading 1’s (so the non-zero row vectors) will form a basis for the row space
of U. The column vectors that contain the leading
1’s from the row vectors will form a basis for the column space of U.
|
|
Example 3 Find
the basis for the row and column space of the following matrix.
Solution
Okay, the basis
for the row space is simply all the row vectors that contain a leading
1. So, for this matrix the basis for
the row space is,
We can also see
that the dimension of the row space will be 3.
The basis for
the column space will be the columns that contain leading 1’s and so for this
matrix the basis for the column space will be,
Note that we subscripted the vectors here with the column
that each came out of. We will
generally do that for these problems.
Also note that the dimension of the column space is 3 as well.
|
Now, all of this is fine provided we have a matrix in
row-echelon form. However, as we know,
most matrices will not be in row-echelon form.
The following two theorems will tell us how to find the basis for the
row and column space of a general matrix.
|
Theorem 2 Suppose
that A is a matrix and U is a matrix in row-echelon form that
has been obtained by performing row operations on A. Then the row space of A and the row space of U are the same space.
|
So, how does this theorem help us? We’ll if the matrix A and U have the same row
space then if we know a basis for one of them we will have a basis for the
other. Notice as well that we assumed
the matrix U is in row-echelon form
and we do know how to find a basis for its row space. Therefore, to find a basis for the row space
of a matrix A we’ll need to reduce it
to row-echelon form. Once in row-echelon
form we can write down a basis for the row space of U, but that is the same as the row space of A and so that set of vectors will also be a basis for the row space
of A.
So, what about a basis for the column space? That’s not quite as straight forward, but is
almost as simple.
|
Theorem 3 Suppose
that A and B are two row equivalent matrices (so we got from one to the
other by row operations) then a set of column vectors from A will be a basis for the column space
of A if and only if the
corresponding columns from B will
form a basis for the column space of B.
|
How does this theorem help us to find a basis for the column
space of a general matrix? We’ll let’s
start with a matrix A and reduce it
to row-echelon form, U, (which we’ll
need for a basis for the row space anyway).
Now, because we arrived at U
by applying row operations to A we
know that A and U are row equivalent. Next,
from Theorem 1 we know how to identify the columns from U that will form a basis for the column space of U.
These columns will probably not be a basis for the column space of A however, what Theorem 3 tells us is
that corresponding columns from A
will form a basis for the columns space of A. For example, suppose the columns 1, 2, 5 and
8 from U form a basis for the column
space of U then columns 1, 2, 5 and 8
from A will form a basis for the
column space of A.
Before we work an example we can now talk about the
dimension of the row and column space of a matrix A. From our theorems above
we know that to find a basis for both the row and column space of a matrix A we first need to reduce it to
row-echelon form and we can get a basis for the row and column space from
that.
Let’s go back and take a look at Theorem 1 in a little more
detail. According to this theorem the
rows with leading 1’s will form a basis for the row space and the columns that
containing the same leading 1’s will form a basis for the column space. Now, there are a fixed number of leading 1’s
and each leading 1 will be in a separate column. For example, there won’t be two leading 1’s
in the second column because that would mean that the upper 1 (one) would not
be a leading 1.
Think about this for a second. If there are k leading 1’s in a row-echelon matrix then there will be k row vectors in a basis for the row
space and so the row space will have a dimension of k. However, since each of
the leading 1’s will be in separate columns there will also be k column vectors that will form a basis
for the column space and so the column space will also have a dimension of k.
This will always happen and this is the reason that we delayed talking
about the dimension of the row and column space above. We needed to get a couple of theorems out of
the way so we could give the following theorem/definition.
|
Theorem 4 Suppose
that A is a matrix then the row
space of A and the column space of A will have the same dimension. We call this common dimension the rank of A and denote it by .
|
Note that if A is
an matrix we know that the row space will be a
subspace of and hence have a dimension of m or less and that the column space will
be a subspace of and hence have a dimension of n or less. Then, because we know that the dimension of
the row and column space must be the same we have the following upper bound for
the rank of a matrix.
We should now work an example.
|
Example 4 Find
a basis for the row and column space of the matrix from Example 2 above. Determine the rank of the matrix.
Solution
Before starting this example let’s note that by the upper
bound for the rank above we know that the largest that the rank can be is 3
since that is the smaller of the number of rows and columns in A.
So, the first thing that we need to do is get the matrix
into row-echelon form. We will leave
it to you to verify that the following is one possible row echelon form for
the matrix from Example 2 above. If
you need a refresher on how to reduce a matrix to row-echelon form you can go
back to the section on Solving Systems of
Equations for a refresher. Also,
recall that there is more than one possible row-echelon form for a given
matrix.
So, a basis for the row space of the matrix will be every
row that contains a leading 1 (all of them in this case). A basis for the row space is then,
Next, the first
three columns of U will form a
basis for the column space of U
since they all contain the leading 1’s.
Therefore the first three columns of A will form a basis for the column space of A. This gives the
following basis for the column space of A.
Now, as Theorem 4 suggested both the row space and the
column space of A have dimension 3 and so we have that
|
Before going on to another example let’s stop for a bit and
take a look at the results of Examples 2 and 4.
From these two examples we saw that the rank and nullity of the matrix
used in those examples were both 3. The
fact that they were the same won’t always happen as we’ll see shortly and so
isn’t all that important. What is
important to note is that and there were 6 columns in this matrix. This in fact will always be the case.
Let’s take a look at a couple more examples now.
|
Example 5 Find
a basis for the null space, row space and column space of the following
matrix. Determine the rank and nullity
of the matrix.
Solution
Before we get
started we can notice that the rank can be at most 4 since that is smaller of the number of
rows and number of columns.
We’ll find the
null space first since that was the first thing asked for. To do this we’ll need to solve the
following system of equations.
You should
verify that the solution is,
The null space
is then given by,
and so we can
see that a basis for the null space is,
Therefore we
now know that . At this point we know the rank of A by Theorem 5 above. According to this theorem the rank must be,
This will give
us a nice check when we find a basis for the row space and the column
space. We now know that each should
contain three vectors.
Speaking of
which, let’s get a basis for the row space and the column space. We’ll need to reduce A to row-echelon form first.
We’ll leave it to you to verify that a possible row-echelon form for A is,
The rows
containing leading 1’s will form a basis for the row space of A and so this basis is,
Next, the
first, second and fourth columns of U
contain leading 1’s and so will form a basis for the column space of U and this tells us that the first,
second and fourth columns of A will
form a basis for the column space of A. Here is that basis.
Note that the dimension of each of these is 3 as we noted
it should be above.
|