Paul's Online Math Notes
     
 
Online Notes / Linear Algebra / Vector Spaces / QR-Decomposition
Linear Algebra

You can navigate through this E-Book using the menu to the left. For E-Books that have a Chapter/Section organization each option in the menu to the left indicates a chapter and will open a menu showing the sections in that chapter. Alternatively, you can navigate to the next/previous section or chapter by clicking the links in the boxes at the very top and bottom of the material.

Also, depending upon the E-Book, it will be possible to download the complete E-Book, the chapter containing the current section and/or the current section. You can do this be clicking on the E-Book, Chapter, and/or the Section link provided below.

For those pages with mathematics on them you can, in most cases, enlarge the mathematics portion by clicking on the equation. Click the enlarged version to hide it.

In this section we’re going to look at a way to “decompose” or “factor” an  matrix as follows.

 

Theorem 1 Suppose that A is an  matrix with linearly independent columns then A can be factored as,

                                                                   

where Q is an  matrix with orthonormal columns and R is an invertible  upper triangular matrix.

 

Proof : The proof here will consist of actually constructing Q and R and showing that they in fact do multiply to give A.

 

Okay, let’s start with A and suppose that it’s columns are given by , , … , .  Also suppose that we perform the Gram-Schmidt process on these vectors and arrive at a set of orthonormal vectors , , … , .  Next, define Q (yes, the Q in the theorem statement) to be the  matrix whose columns are , , … ,  and so Q will be a matrix with orthonormal columns.  We can then write A and Q as,

 

 

Next, because each of the  ’s are in  we know from Theorem 2 of the previous section that we can write each  as a linear combination of , , … ,  in the following manner.

 

 

 

Next, define R (and yes, this will eventually be the R from the theorem statement) to be the  matrix defined as,

 

 

 

Now, let’s examine the product, QR.

 

 

 

From the section on Matrix Arithmetic we know that the jth column of this product is simply Q times the jth column of R.  However, if you work through a couple of these you’ll see that when we multiply Q times the jth column of R we arrive at the formula for  that we’ve got above.  In other words,

 

 

 

So, we can factor A as a product of Q and R and Q has the correct form.  Now all that we need to do is to show that R is an invertible upper triangular matrix and we’ll be done.  First, from the Gram-Schmidt process we know that  is orthogonal to , , … , .  This means that all the inner products below the main diagonal must be zero since they are all of the form  with .

 

Now, we know from Theorem 2 from the Special Matrices section that a triangular matrix will be invertible if the main diagonal entries, , are non-zero.  This is fairly easy to show.  Here is the general formula for  from the Gram-Schmidt process.

 

 

 

Recall that we’re assuming that we found the orthonormal  ’s and so each of these will have a norm of 1 and so the norms are not needed in the formula.  Now, solving this for  gives,

 

 

 

Let’s look at the diagonal entries of R.  We’ll plug in the formula for  into the inner product and do some rewriting using the properties of the inner product.

 

 

 

However the  are orthonormal basis vectors and so we know that

 

 

 

Using these we see that the diagonal entries are nothing more than,

 

 

 

So, the diagonal entries of R are non-zero and hence R must be invertible.

Pf_Box

 

So, now that we’ve gotten the proof out of the way let’s work an example.

 

Example 1  Find the QR-decomposition for the matrix,

                                                          

Solution

The columns from A are,

                                          

 

We performed Gram-Schmidt on these vectors in Example 3 of the previous section.  So, the orthonormal vectors that we’ll use for Q are,

                              

and the matrix Q is,

                                                     

 

The matrix R is,

                              

 

So, the QR-Decomposition for this matrix is,

                            

 

We’ll leave it to you to verify that this multiplication does in fact give A.