You can navigate through this E-Book using the menu to the left. For E-Books that have a Chapter/Section organization each option in the menu to the left indicates a chapter and will open a menu showing the sections in that chapter. Alternatively, you can navigate to the next/previous section or chapter by clicking the links in the boxes at the very top and bottom of the material.
Also, depending upon the E-Book, it will be possible to download the complete E-Book, the chapter containing the current section and/or the current section. You can do this be clicking on the E-Book, Chapter, and/or the Section link provided below.
For those pages with mathematics on them you can, in most cases, enlarge the mathematics portion by clicking on the equation. Click the enlarged version to hide it.
We now need to come back and revisit the topic of basis. We are going
to be looking at a special kind of basis in this section that can arise in an
inner product space, and yes it does require an inner product space to
construct. However, before we do that
we’re going to need to get some preliminary topics out of the way first.
We’ll first need to get a set of definitions out of way.
|
Definition 1 Suppose
that S is a set of vectors in an
inner product space.
(a) If
each pair of distinct vectors from S
is orthogonal then we
call S an orthogonal set.
(b) If
S is an orthogonal set and each of
the vectors in S also has a norm of
1 then we call S an orthonormal set.
|
Let’s take a quick look at an example.
|
Example 1 Given
the three vectors  ,
and in answer each of the following.
(a) Show
that they form an orthogonal set under the standard Euclidean inner product
for but not an orthonormal set. [Solution]
(b) Turn
them into a set of vectors that will form an orthonormal set of vectors under
the standard Euclidean inner product for . [Solution]
Solution
(a) Show that they form an
orthogonal set under the standard Euclidean inner product for but not an orthonormal set.
All we need to do here to show that they form an
orthogonal set is to compute the inner product of all the possible pairs and
show that they are all zero.
So, they do
form an orthogonal set. To show that
they don’t form an orthonormal set we just need to show that at least one of
them does not have a norm of 1. For
the practice we’ll compute all the norms.
So, one of them
has a norm of 1, but the other two don’t and so they are not an orthonormal
set of vectors.
[Return to Problems]
(b) Turn them into a set of vectors that will
form an orthonormal set of vectors under the standard Euclidean inner product
for .
We’ve actually
done most of the work here for this part of the problem already. Back when we were working in we saw that we could turn any vector v into a vector with norm 1 by
dividing by its norm as follows,
This new vector
will have a norm of 1. So, we can turn
each of the vectors above into a set of vectors with norm 1.
All that
remains is to show that this new set of vectors is still orthogonal. We’ll leave it to you to verify that,
and so we have turned the three vectors into a set of
vectors that form an orthonormal set.
[Return to Problems]
|
We have the following very nice fact about orthogonal sets.
|
Theorem 1 Suppose
is an orthogonal set of non-zero vectors in
an inner product space, then S is
also a set of linearly independent vectors.
|
Proof : Note that
we need the vectors to be non-zero vectors because the zero vector could be in
a set of orthogonal vectors and yet we know that if a set includes the
zero vector it will be linearly dependent.
So, now that we know there is a chance that these vectors
are linearly independent (since we’ve excluded the zero vector) let’s form the
equation,
and we’ll need to show that the only scalars that work here
are ,
,
… , .
In fact, we can do this in a single step. All we need to do is take the inner product
of both sides with respect to ,
,
and then use the properties of inner products to rearrange things a little.
Now, because we know the vectors in S are orthogonal we know that if and so this reduced down to,
Next, since we know that the vectors are all non-zero we
have and so the only way that this can be zero is
if . So, we’ve shown that we must have ,
,
… , and so these vectors are linearly independent.
Okay, we are now ready to move into the main topic of this
section. Since a set of orthogonal
vectors are also linearly independent if they just happen to span the vector
space we are working on they will also form a basis for the vector space.
|
Definition 2 Suppose
that is a basis for an inner product space.
(a) If
S is also an orthogonal set then we
call S an orthogonal basis.
(b) If
S is also an orthonormal set then
we call S an orthonormal basis.
|
Note that we’ve been using an orthonormal basis already to
this point. The standard basis vectors for are an orthonormal basis.
The following fact gives us one of the very nice properties
about orthogonal/orthonormal basis.
|
Theorem 2 Suppose
that is an orthogonal basis for an inner product
space and that u is any vector
from the inner product space then,
If in addition S is in fact an orthonormal basis
then,
|
Proof : We’ll
just show that the first formula holds.
Once we have that the second will follow directly from the fact that all
the vectors in an orthonormal set have a norm of 1.
So, given u we
need to find scalars ,
,
… , so that,
To find these scalars simply take the inner product of both
sides with respect to ,
.
Now, since we have an orthogonal basis we know that if and so this reduces to,
Also, because is a basis vector we know that it isn’t the
zero vector and so we also know that . This then gives us,
However, from the definition of the norm we
see that we can also write this as,
and so we’re done.
What this theorem is telling us is that for any vector in an
inner product space, with an orthogonal/orthonormal basis, it is very easy to
write down the linear combination of basis vectors for that vector. In other words, we don’t need to go through
all the work to find the linear combinations that we were doing in earlier
sections.
We would like to be able to construct an
orthogonal/orthonormal basis for a finite dimensional vector space given any
basis of that vector space. The
following two theorems will help us to do that.
Note that this theorem is really an extension of the idea of
projections that we saw when we
first introduced the concept of the dot product. Also note that can be easily computed from by,
This theorem is not really the one that we need to construct
an orthonormal basis. We will use
portions of this theorem, but we needed it more to acknowledge that we could do
projections and to get the notation out of the way. The following theorem is the one that will be
the main workhorse of the process.
|
Theorem 4 Suppose
that W is a finite dimensional
subspace of an inner product space V. Further suppose that W has an orthogonal basis and that u is any vector in V
then,
If in addition S is in fact an orthonormal basis
then,
|
So, just how does this theorem help us to construct an
orthogonal/orthonormal basis? The
following process, called the Gram-Schmidt
process, will construct an orthogonal/orthonormal basis for a finite
dimensional inner product space given any basis. We’ll also be able to develop some very nice
facts about the basis that we’re going to be constructing as we go through the
construction process.
Gram-Schmidt Process
Suppose that V is
a finite dimensional inner product space and that is a basis for V. The following process
will construct an orthogonal basis for V,
. To find an orthonormal basis simply divide
the ’s by their norms.
Step 1 : Let .
Step 2 : Let and then define (i.e.
is the portion of that is orthogonal to ).
Technically, this is all there is to step 2 (once we show that anyway) since will be orthogonal to because it is in . However, this isn’t terribly useful from a
computational standpoint. Using the
result of Theorem 3 and the formula from Theorem 4 gives us the following
formula for ,
Next, we need to verify that because the zero vector cannot be a basis
vector. To see that assume for a second that we do have . This would give us,
But this tells us that is a multiple of which we can’t have since they are both basis
vectors and are hence linearly independent.
So, .
Finally, let’s observe an interesting consequence of how we
found . Both and are orthogonal and so are linearly independent
by Theorem 1 above and this means that they are a basis for the subspace and this subspace has dimension of 2. However, they are also linear combinations of
and and so is a subspace of which also has dimension 2. Therefore, by Theorem 9 from the section on Basis we can
see that we must in fact have,
So, the two new vectors, and ,
will in fact span the same subspace as the two original vectors, and ,
span. This is a nice consequence of the
Gram-Schmidt process.
Step 3 : This
step is really an extension of Step 2 and so we won’t go into quite the detail
here as we did in Step 2. First, define and then define and so will be the portion of that is orthogonal to both and .
We can compute as follows,
Next, both and are linear combinations of and and so can be thought of as a linear combination of ,
,
and . Then because ,
,
and are linearly independent we know that we must
have . You should probably go through the steps of
verifying the claims made here for the practice.
With this step we can also note that because is in the orthogonal complement of (by construction) and because we know that,
from the previous step we know as well that must be orthogonal to all vectors in . In particular must be orthogonal to and .
Finally, following an argument similar to that in Step 2 we
get that,
Step 4 : Continue
in this fashion until we’ve found .
There is the Gram-Schmidt process. Going through the process above, with all the
explanation as we provided, can be a little daunting and can make the process
look more complicated than it in fact is.
Let’s summarize the process before we go onto a couple of examples.
Gram-Schmidt Process
Okay, let’s go through a couple of examples here.
|
Example 2 Given
that ,
,
and is a basis of and assuming that we’re working with the
standard Euclidean inner product construct an orthogonal basis for .
Solution
You should verify that the set of vectors above is in fact
a basis for . Now, we’ll need to go through the
Gram-Schmidt process a couple of times.
The first step is easy.
The remaining two steps are going to involve a little more
work, but won’t be all that bad. Here
is the formula for the second vector in our orthogonal basis.
and here is all
the quantities that we’ll need.
The second
vector is then,
The formula for
the third (and final vector) is,
and here are
the quantities that we need for this step.
The third
vector is then,
So, the
orthogonal basis that we’ve constructed is,
You should verify that these do in fact form an orthogonal
set.
|
|
Example 3 Given
that ,
,
and is a basis of and assuming that we’re working with the
standard Euclidean inner product construct an orthonormal basis for .
Solution
First, note that this is almost the same problem as the
previous one except this time we’re looking for an orthonormal basis instead of an orthogonal basis. There
are two ways to approach this. The
first is often the easiest way and that is to acknowledge that we’ve got a
orthogonal basis and we can turn that into an orthonormal basis simply by
dividing by the norms of each of the vectors.
Let’s do it this way and see what we get.
Here are the norms of the vectors from the previous
example.
Note that in
order to eliminate as many square roots as possible we rationalized the
denominators of the fractions here.
Dividing by the
norms gives the following set of vectors.
Okay that’s the first way to do it. The second way is to go through the
Gram-Schmidt process and this time divide by the norm as we find each new
vector. This will have two
effects. First, it will put a fair
amount of roots into the vectors that we’ll need to work with. Second, because we are turning the new
vectors into vectors with length one the norm in the Gram-Schmidt formula
will also be 1 and so isn’t needed.
Let’s go through this once just to show you the
differences.
The first new vector will be,
Now, to get the
second vector we first need to compute,
however we
won’t call it yet since we’ll need to divide by it’s norm
once we’re done. Also note that we’ve
acknowledged that the norm of is 1 and so we don’t need that in the
formula. Here is the dot product that
we need for this step.
Here is the new
orthogonal vector.
Notice that
this is the same as the second vector we found in Example 2. In this case we’ll need to divide by its
norm to get the vector that we want in this case.
Finally, for
the third orthogonal vector the
formula will be,
and again we’ve
acknowledged that the norms of the first two vectors will be 1 and so aren’t
needed in this formula. Here are the
dot products that we’ll need.
The orthogonal vector is then,
Again, this is
the third orthogonal vector that we
found in Example 2. Here is the final
step to get our third orthonormal vector for this problem.
So, we got exactly the same vectors as if we did when we
just used the results of Example 2. Of
course that is something that we should expect to happen here.
|
So, as we saw in the previous example there are two ways to
get an orthonormal basis from any given basis.
Each has its pros and cons and you’ll need to decide which method to
use. If we first compute the orthogonal
basis and the divide all of them at then end by their norms we don’t have to
work much with square roots, however we do need to compute norms that we won’t
need otherwise. Again, it will be up to
you to determine what the best method for you to use is.
|
Example 4 Given
that ,
,
and is a basis of and assuming that we’re working with the
standard Euclidean inner product construct an orthonormal basis for .
Solution
Now, we’re looking for an orthonormal basis and so we’ve
got our two options on how to proceed here.
In this case we’ll construct an orthogonal basis and then convert that
into an orthonormal basis at the very end.
The first vector is,
Here’s the dot
product and norm we need for the second vector.
The second
orthogonal vector is then,
For the third
vector we’ll need the following dot products and norms
and the third
orthogonal vector is,
Finally, for
the fourth orthogonal vector we’ll need,
and the fourth
vector in out new orthogonal basis is,
Okay, the
orthogonal basis is then,
Next, we’ll
need their norms so we can turn this set into an orthonormal basis.
The orthonormal
basis is then,
|
Now, we saw how to
expand a linearly independent set of vectors into a basis for a vector
space. We can do the same thing here
with orthogonal sets of vectors and the Gram-Schmidt process.
|
Example 5 Expand
the vectors and into an orthogonal basis for and assume that we’re working with the
standard Euclidean inner product.
Solution
First notice that the two vectors are already orthogonal
and linearly independent. Since they
are linearly independent and we know that a basis for will contain 3 vectors we know that we’ll
only need to add in one more vector.
Next, since they are already orthogonal that will simplify some of the
work.
Now, recall that in order to expand a linearly independent
set into a basis for a vector space we need to add in a vector that is not in
the span of the original vectors.
Doing so will retain the linear independence of the set. Since both of these vectors have a zero in
the second term we can add in any of the following to the set.
If we used the
first one we’d actually have an orthogonal set without any work, but that
would be boring and defeat the purpose of the example. To make our life at least somewhat easier
with the work let’s add in the fourth on to get the set of vectors.
Now, we know
these are linearly independent and since there are three vectors by Theorem 6 from the section on Basis we
know that they form a basis for .
However, they don’t form an orthogonal basis.
To get an
orthogonal basis we would need to perform Gram-Schmidt on the set. However, since the first two vectors are
already orthogonal performing Gram-Schmidt would not have any affect (you
should verify this). So, let’s just
rename the first two vectors as,
and then just
perform Gram-Schmidt for the third vector.
Here are the dot products and norms that we’ll need.
The third
vector will then be,
|