In this section we want to take a look at some important subspaces that are associated with matrices.  In fact they are so important that they are often called the fundamental subspaces of a matrix.  We’ve actually already seen one of the fundamental subspaces, the null space, previously although we will give its definition here again for the sake of completeness. 

 

Before we give the formal definitions of the fundamental subspaces we need to quickly review a concept that we first saw back when we were looking at matrix arithmetic.

 

Given an  matrix

The row vectors (we called them row matrices at the time) are the vectors in  formed out of the rows of A.  The column vectors (again we called them column matrices at the time) are the vectors in  that are formed out of the columns of A.

 

Example 1  Write down the row vectors and column vectors for                                                                Solution The row vectors are,            The column vectors are

 

Note that despite the fact that we’re calling them vectors we are using matrix notation for them.  The reason is twofold.  First, they really are row/column matrices and so we may as well denote them as such and second in this way we can keep the “orientation” of each to remind us whether or not they are row vectors or column vectors.  In other words, row vectors are listed horizontally and column vectors are listed vertically.

 

Because we’ll be using the matrix notation for the row and column vectors we’ll be using matrix notation for vectors in general in this section so we won’t be mixing and matching the notations too much.

 

Here then are the definitions of the three fundamental subspaces that we’ll be investigating in this section.

 

Definition 1  Suppose that A is an  matrix. (a) The subspace of  that is spanned by the row vectors of A is called the row space of A. (b) The subspace of  that is spanned by the column vectors of A is called the column space of A. (c) The set of all x in   such that  (which is a subspace of  by Theorem 2 from the Subspaces section) is called the null space of A.

 

We are going to be particularly interested in the basis for each of these subspaces and that in turn means that we’re going to be able to discuss the dimension of each of them.  At this point we can give the notation for the dimension of the null space, but we’ll need to wait a bit before we do so for the row and column spaces.  The reason for the delay will be apparent once we reach that point.  So, let’s go ahead and give the notation for the null space.

 

Definition 2  The dimension of the null space of A is called the nullity of A and is denoted by .

 

We should work an example at this point.  Because we’ve already seen how to find the basis for the null space (Example 4(b) in the Subspaces section and Example 7 of the Basis section) we’ll do one example at this point and then devote the remainder of the discussion on basis/dimension of these subspaces to finding the basis/dimension for the row and column space.  Note that we will see an example or two later in this section of null spaces as well.

 

Example 2  Determine a basis for the null space of the following matrix.                                                Solution So, to find the null space we need to solve the following system of equations.                                              We’ll leave it to you to verify that the solution is given by,                    In matrix form the solution can be written as,                                           So, the solution can be written as a linear combination of the three linearly independent vectors (verify the linearly independent claim!)                                      and so these three vectors then form the basis for the null space since they span the null space and are linearly independent.  Note that this also means that the null space has a dimension of 3 since there are three basis vectors for the null space and so we can see that

 

Again, remember that we’ll be using matrix notation for vectors in this section.

 

Okay, now that we’ve gotten an example of the basis for the null space taken care of we need to move onto finding bases (and hence the dimensions) for the row and column spaces of a matrix.  However, before we do that we first need a couple of theorems out of the way.  The first theorem tells us how to find the basis for a matrix that is in row-echelon form.

 

Theorem 1  Suppose that the matrix U is in row-echelon form.  The row vectors containing leading 1’s (so the non-zero row vectors) will form a basis for the row space of U.  The column vectors that contain the leading 1’s from the row vectors will form a basis for the column space of U.

 

Example 3  Find the basis for the row and column space of the following matrix.                                                    Solution Okay, the basis for the row space is simply all the row vectors that contain a leading 1.  So, for this matrix the basis for the row space is,                       We can also see that the dimension of the row space will be 3.   The basis for the column space will be the columns that contain leading 1’s and so for this matrix the basis for the column space will be,                                             Note that we subscripted the vectors here with the column that each came out of.  We will generally do that for these problems.  Also note that the dimension of the column space is 3 as well.

 

Now, all of this is fine provided we have a matrix in row-echelon form.  However, as we know, most matrices will not be in row-echelon form.  The following two theorems will tell us how to find the basis for the row and column space of a general matrix.

 

Theorem 2  Suppose that A is a matrix and U is a matrix in row-echelon form that has been obtained by performing row operations on A.  Then the row space of A and the row space of U are the same space.

 

So, how does this theorem help us?  Well if the matrix A and U have the same row space then if we know a basis for one of them we will have a basis for the other.  Notice as well that we assumed the matrix U is in row-echelon form and we do know how to find a basis for its row space.  Therefore, to find a basis for the row space of a matrix A we’ll need to reduce it to row-echelon form.  Once in row-echelon form we can write down a basis for the row space of U, but that is the same as the row space of A and so that set of vectors will also be a basis for the row space of A.

 

So, what about a basis for the column space?  That’s not quite as straight forward, but is almost as simple.

 

Theorem 3  Suppose that A and B are two row equivalent matrices (so we got from one to the other by row operations) then a set of column vectors from A will be a basis for the column space of A if and only if the corresponding columns from B will form a basis for the column space of B.

 

How does this theorem help us to find a basis for the column space of a general matrix?  We’ll let’s start with a matrix A and reduce it to row-echelon form, U, (which we’ll need for a basis for the row space anyway).  Now, because we arrived at U by applying row operations to A we know that A and U are row equivalent.  Next, from Theorem 1 we know how to identify the columns from U that will form a basis for the column space of U.  These columns will probably not be a basis for the column space of A however, what Theorem 3 tells us is that corresponding columns from A will form a basis for the column space of A.  For example, suppose the columns 1, 2, 5 and 8 from U form a basis for the column space of U then columns 1, 2, 5 and 8 from A will form a basis for the column space of A.

 

Before we work an example we can now talk about the dimension of the row and column space of a matrix A.  From our theorems above we know that to find a basis for both the row and column space of a matrix A we first need to reduce it to row-echelon form and we can get a basis for the row and column space from that. 

 

Let’s go back and take a look at Theorem 1 in a little more detail.  According to this theorem the rows with leading 1’s will form a basis for the row space and the columns that containing the same leading 1’s will form a basis for the column space.  Now, there are a fixed number of leading 1’s and each leading 1 will be in a separate column.  For example, there won’t be two leading 1’s in the second column because that would mean that the upper 1 (one) would not be a leading 1.

 

Think about this for a second.  If there are k leading 1’s in a row-echelon matrix then there will be k row vectors in a basis for the row space and so the row space will have a dimension of k.  However, since each of the leading 1’s will be in separate columns there will also be k column vectors that will form a basis for the column space and so the column space will also have a dimension of k.  This will always happen and this is the reason that we delayed talking about the dimension of the row and column space above.  We needed to get a couple of theorems out of the way so we could give the following theorem/definition.

 

Theorem 4  Suppose that A is a matrix then the row space of A and the column space of A will have the same dimension.  We call this common dimension the rank of A and denote it by .

 

Note that if A is an  matrix we know that the row space will be a subspace of  and hence have a dimension of m or less and that the column space will be a subspace of  and hence have a dimension of n or less.  Then, because we know that the dimension of the row and column space must be the same we have the following upper bound for the rank of a matrix.

We should now work an example.

 

Example 4  Find a basis for the row and column space of the matrix from Example 2 above.  Determine the rank of the matrix.   Solution Before starting this example let’s note that by the upper bound for the rank above we know that the largest that the rank can be is 3 since that is the smaller of the number of rows and columns in A.   So, the first thing that we need to do is get the matrix into row-echelon form.  We will leave it to you to verify that the following is one possible row echelon form for the matrix from Example 2 above.  If you need a refresher on how to reduce a matrix to row-echelon form you can go back to the section on Solving Systems of Equations for a refresher.  Also, recall that there is more than one possible row-echelon form for a given matrix.                                                   So, a basis for the row space of the matrix will be every row that contains a leading 1 (all of them in this case).  A basis for the row space is then,             Next, the first three columns of U will form a basis for the column space of U since they all contain the leading 1’s.  Therefore the first three columns of A will form a basis for the column space of A.  This gives the following basis for the column space of A.                                 Now, as Theorem 4 suggested both the row space and the column space of A have dimension 3 and so we have that

 

Before going on to another example let’s stop for a bit and take a look at the results of Examples 2 and 4.  From these two examples we saw that the rank and nullity of the matrix used in those examples were both 3.  The fact that they were the same won’t always happen as we’ll see shortly and so isn’t all that important.  What is important to note is that  and there were 6 columns in this matrix.  This in fact will always be the case.

 

Theorem 5  Suppose that A is an  matrix.  Then,

 

Let’s take a look at a couple more examples now.

 

Example 5  Find a basis for the null space, row space and column space of the following matrix.  Determine the rank and nullity of the matrix.                                               Solution Before we get started we can notice that the rank can be at most  4 since that is smaller of the number of rows and number of columns.   We’ll find the null space first since that was the first thing asked for.  To do this we’ll need to solve the following system of equations.                                                     You should verify that the solution is,                The null space is then given by,                                                     and so we can see that a basis for the null space is,                                                   Therefore we now know that .  At this point we know the rank of A by Theorem 5 above.  According to this theorem the rank must be,                                           This will give us a nice check when we find a basis for the row space and the column space.  We now know that each should contain three vectors.   Speaking of which, let’s get a basis for the row space and the column space.  We’ll need to reduce A to row-echelon form first.  We’ll leave it to you to verify that a possible row-echelon form for A is,                                                      The rows containing leading 1’s will form a basis for the row space of A and so this basis is,                        Next, the first, second and fourth columns of U contain leading 1’s and so will form a basis for the column space of U and this tells us that the first, second and fourth columns of A will form a basis for the column space of A.  Here is that basis.                                Note that the dimension of each of these is 3 as we noted it should be above.

 

Example 6  Find a basis for the null space, row space and column space of the following matrix.  Determine the nullity and rank of this matrix.                                                                Solution In this case we can notice that the rank of this matrix can be at most 2 since that is the minimum of the number of rows and number of columns.   To find the null space we’ll need to solve the following system of equations,                                                                We’ll leave it to you to verify that the solution to this system is,                                                          This is actually the point to this problem.  There is only a single solution to the system above, namely the zero vector, 0.  Therefore the null space consists solely of the zero vector and vector spaces that consist solely of the zero vector do not have a basis and so we can’t give one.  Also, vector spaces consisting solely of the zero vectors are defined to have a dimension of zero.  Therefore, the nullity of this matrix is zero.  This also tells us that the rank of this matrix must be 2 by Theorem 5.   Let’s now find a basis for the row space and the column space.  You should verify that one possible row-reduced form for A is,                                                               A basis for the row space of A is then,                                                and since both columns of U form a basis for the column space of U both columns from A will form a basis for the column space of A.  The basis for the column space of A is then,                                                 Once again, both have dimension of 2 as we knew they should from our use of Theorem 5 above.

 

In all of the examples that we’ve worked to this point in finding a basis for the row space and the column space we should notice that the basis we found for the column space consisted of columns from the original matrix while the basis we found for the row space did not consist of rows from the original matrix.

 

Also note that we can’t necessarily use the same idea we used to get a basis for the column space to get a basis for the row space.  For example let’s go back and take a look at Example 5.  The first three rows of U formed a basis for the row space, but that does not mean that the first three rows of A will also form a basis for the row space.  In fact, in this case they won’t.  In this case the third row is twice the first row added onto the second row and so the first three rows are not linearly independent (which you’ll recall is required for a set of vectors to be a basis).

 

So, what do we do if we do want rows from the original matrix to form our basis?  The answer to this is surprisingly simple. 

 

Example 7  Find a basis for the row space of the matrix in Example 5 that consists of rows from the original matrix.   Solution The first thing that we’ll do is take the transpose of A.  In doing so the rows of A will become the columns of .  This means that the row space of A will become the column space of .  Recall as well that we find a basis for the column space in terms of columns from the original matrix ( in this case).  So, we’ll be finding a basis for the column space of  in terms of the columns of , but the columns of  are the rows of A and the column space of  is the row space of A.  Therefore, when this is all said and done by finding a basis for the column space of  we will also be finding a basis for the row space of A and it will be in terms of rows from A and not rows from the row-echelon form of the matrix.   So, here is the transpose of A.                                                     Here is a possible row-echelon form of the transpose (you should verify this).                                                         The first, second and fourth columns of U form a basis for the column space of U and so a basis for the column space of  is,                                  Again, however, the column space of  is nothing more than the row space of A and so these three column are rows from A and will also form a basis for the row space.  So, let’s change notation a little to make it clear that we’re dealing with a basis for the row space and we’ll be done.  Here is a basis for the row space of A in terms of rows from A itself.

 

Next we want to give a quick theorem that gives a relationship between the solution to a system of equations and the column space of the coefficient matrix.  This theorem can be useful on occasion.

 

Theorem 6  The system of linear equations  will be consistent (i.e. have at least one solution) if and only if b is in the column space of A.

 

Note that since the basis for the column space of a matrix is given in terms of the certain columns of A this means that a system of equations will be consistent if and only if b can be written as a linear combination of at least some of the columns of A.  This should be clear from application of the Theorem above.  This theorem tells us that b must be in the column space of A, but that means that it can be written as a linear combination of the basis vectors for the column space of A.

 

We’ll close out this section with a couple of theorems relating the invertibility of a square matrix A to some of the ideas in this section.

 

Theorem 7  Let A be an  matrix.  The following statements are equivalent. (a) A is invertible. (b) The null space of A is , i.e. just the zero vector. (c) . (d) . (e) The columns vectors of A form a basis for . (f) The row vectors of A form a basis for .

 

The proof of this theorem follows directly from Theorem 9 in the Properties of Determinants section and from the definitions of null space, rank and nullity so we’re not going to give it here.  We will point out however that if the rank of an  matrix is n then a basis for the row (column) space must contain n vectors, but there are only n rows (columns) in A and so all the rows (columns) of A must be in the basis.  Also, the row (column) space is a subspace of  which also has a dimension of n.  These ideas are helpful in showing that (d) will imply either (e) or (f).

 

Finally, speaking of Theorem 9 in the Properties of Determinant section, this was also a theorem listing many equivalent statements on the invertibility of a matrix.  We can merge that theorem with Theorem 7 above into the following theorem.

 

Theorem 8  Let A be an  matrix.  The following statements are equivalent. (a) A is invertible. (b) The only solution to the system  is the trivial solution. (c) A is row equivalent to . (d) A is expressible as a product of elementary matrices. (e)  has exactly one solution for every  matrix b. (f)  is consistent for every  matrix b. (g)   (h) The null space of A is , i.e. just the zero vector. (i) . (j) . (k) The columns vectors of A form a basis for . (l) The row vectors of A form a basis for .