Review : Matrices and
Vectors
This section is intended to be a catch all for many of the
basic concepts that are used occasionally in working with systems of
differential equations. There will not
be a lot of details in this section, nor will we be working large numbers of
examples. Also, in many cases we will
not be looking at the general case since we won’t need the general cases in our
differential equations work.
Let’s start with some of the basic notation for
matrices. An n x m (this is often
called the size or dimension of the matrix) matrix is a
matrix with n rows and m columns and the entry in the i^{th} row and j^{th} column is denoted by a_{ij}. A short hand method of writing a general n x m
matrix is the following.
The size or dimension of a matrix is subscripted as shown if
required. If it’s not required or clear
from the problem the subscripted size is often dropped from the matrix.
Special Matrices
There are a few “special” matrices out there that we may use
on occasion. The first special matrix is
the square matrix. A square matrix is any matrix whose size (or
dimension) is n x n.
In other words it has the same number of rows as columns. In a square matrix the diagonal that starts
in the upper left and ends in the lower right is often called the main diagonal.
The next two special matrices that we want to look at are
the zero matrix and the identity matrix.
The zero matrix, denoted 0_{n }_{x m} , is a matrix all of whose
entries are zeroes. The identity matrix is a square n x n
matrix, denoted I_{n}, whose
main diagonals are all 1’s and all the other elements are zero. Here are the general zero and identity
matrices.
In matrix arithmetic these two matrices will act in matrix
work like zero and one act in the real number system.
The last two special matrices that we’ll look at here are
the column matrix and the row matrix. These are matrices that consist of a single
column or a single row. In general they
are,
We will often refer to these as vectors.
Arithmetic
We next need to take a look at arithmetic involving
matrices. We’ll start with addition and subtraction of two matrices.
So, suppose that we have two n
x m matrices, A and B. The sum (or difference) of these two matrices
is then,
The sum or difference of two matrices of the same size is a
new matrix of identical size whose entries are the sum or difference of the
corresponding entries from the original two matrices. Note that we can’t add or subtract entries
with different sizes.
Next, let’s look at scalar
multiplication. In scalar
multiplication we are going to multiply a matrix A by a constant (sometimes called a scalar) α. In
this case we get a new matrix whose entries have all been multiplied by the
constant, α.
Example 1 Given
the following two matrices,
compute A5B.
Solution
There isn’t much to do here other than the work.
We first multiplied all the entries of B by 5 then subtracted corresponding
entries to get the entries in the new matrix.

The final matrix operation that we’ll take a look at is matrix multiplication. Here we will start with two matrices, A_{n }_{x p} and B_{p }_{x m}
. Note that A must have the same number of columns as B has rows. If this isn’t
true then we can’t perform the multiplication.
If it is true then we can perform the following multiplication.
The new matrix will have size n x m and the entry in
the i^{th} row and j^{th} column, c_{ij}, is found by multiplying row i of matrix A by column j of matrix B.
This doesn’t always make sense in words so let’s look at an example.
Example 2 Given
compute AB.
Solution
The new matrix will have size 2 x 4. The entry in row 1 and column 1 of the new
matrix will be found by multiplying row 1 of A by column 1 of B. This means that we multiply corresponding
entries from the row of A and the
column of B and then add the results
up. Here are a couple of the entries
computed all the way out.
Here’s the complete solution.

In this last example notice that we could not have done the
product BA since the number of
columns of B does not match the
number of row of A. It is important to note that just because we
can compute AB doesn’t mean that we
can compute BA. Likewise, even if we can compute both AB and BA they may or may not be the same matrix.
Determinant
The next topic that we need to take a look at is the determinant of a matrix. The determinant is actually a function that
takes a square matrix and converts it into a number. The actual formula for the function is
somewhat complex and definitely beyond the scope of this review.
The main method for computing determinants of any square
matrix is called the method of
cofactors. Since we are going to be
dealing almost exclusively with 2 x 2 matrices and the occasional 3 x 3 matrix
we won’t go into the method here. We can
give simple formulas for each of these cases.
The standard notation for the determinant of the matrix A is.
Here are the formulas for the determinant of 2 x 2 and 3 x 3
matrices.
Example 3 Find
the determinant of each of the following matrices.
Solution
For the 2 x 2 there isn’t much to do other than to plug it
into the formula.
For the 3 x 3 we could plug it into the formula, however
unlike the 2 x 2 case this is not an easy formula to remember. There is an easier way to get the same
result. A quicker way of getting the
same result is to do the following.
First write down the matrix and tack a copy of the first two columns
onto the end as follows.
Now, notice that there are three diagonals that run from
left to right and three diagonals that run from right to left. What we do is multiply the entries on each
diagonal up and the if the diagonal runs from left to right we add them up
and if the diagonal runs from right to left we subtract them.
Here is the work for this matrix.

You can either use the formula or the short cut to get the
determinant of a 3 x 3.
If the determinant of a matrix is zero we call that matrix singular and if the determinant of a
matrix isn’t zero we call the matrix nonsingular. The 2 x 2 matrix in the above example was
singular while the 3 x 3 matrix is nonsingular.
Matrix Inverse
Next we need to take a look at the inverse of a matrix. Given a
square matrix, A, of size n x n
if we can find another matrix of the same size, B such that,
then we call B the
inverse of A and denote it by B=A^{1}.
Computing the inverse of a matrix, A, is fairly simple. First
we form a new matrix,
and then use the row operations from the previous section and try to convert this matrix into the form,
If we can then B
is the inverse of A. If we can’t then there is no inverse of the
matrix A.
Example 4 Find
the inverse of the following matrix, if it exists.
Solution
We first form the new matrix by tacking on the 3 x 3
identity matrix to this matrix. This
is
We will now use row operations to try and convert the
first three columns to the 3 x 3 identity.
In other words we want a 1 on the diagonal that starts at the upper
left corner and zeroes in all the other entries in the first three columns.
If you think about it, this process is very similar to the
process we used in the last section to solve
systems, it just goes a little farther.
Here is the work for this problem.
So, we were able to convert the first three columns into
the 3 x 3 identity matrix therefore the inverse exists and it is,

So, there was an example in which the inverse did
exist. Let’s take a look at an example
in which the inverse doesn’t exist.
Example 5 Find
the inverse of the following matrix, provided it exists.
Solution
In this case we will tack on the 2 x 2 identity to get the
new matrix and then try to convert the first two columns to the 2 x 2
identity matrix.
And we don’t need to go any farther. In order for the 2 x 2 identity to be in
the first two columns we must have a 1 in the second entry of the second
column and a 0 in the second entry of the first column. However, there is no way to get a 1 in the
second entry of the second column that will keep a 0 in the second entry in
the first column. Therefore, we can’t
get the 2 x 2 identity in the first two columns and hence the inverse of B doesn’t exist.

We will leave off this discussion of inverses with the
following fact.
Fact
Given a square matrix A.
1. If
A is nonsingular then A^{1} will exist.
2. If
A is singular then A^{1} will NOT exist.

I’ll leave it to you to verify this fact for the previous
two examples.
Systems of Equations
Revisited
We need to do a quick revisit of systems of equations. Let’s start with a general system of
equations.
Now, covert each side into a vector to get,
The left side of this equation can be thought of as a matrix
multiplication.
Simplifying up the notation a little gives,
where, is a vector whose components are the unknowns
in the original system of equations. We
call (2)
the matrix form of the system of equations (1) and
solving (2)
is equivalent to solving (1). The solving process is identical. The augmented matrix for (2)
is
Once we have the augmented matrix we proceed as we did with
a system that hasn’t been written in matrix form.
We also have the following fact about solutions to (2).
Fact
Given the system of equation (2)
we have one of the following three possibilities for solutions.
1. There
will be no solutions.
2. There
will be exactly one solution.
3. There
will be infinitely many solutions.

In fact we can go a little farther now. Since we are assuming that we’ve got the same
number of equations as unknowns the matrix A
in (2)
is a square matrix and so we can compute its determinant. This gives the following fact.
Fact
Given the system of equations in (2)
we have the following.
1. If
A is nonsingular then there will be
exactly one solution to the system.
2. If
A is singular then there will
either be no solution or infinitely many solutions to the system.

The matrix form of a homogeneous system is
where is the vector of all zeroes. In the homogeneous system we are guaranteed
to have a solution, . The fact above for homogeneous systems is
then,
Fact
Given the homogeneous system (3)
we have the following.
1. If
A is nonsingular then the only
solution will be .
2. If
A is singular then there will be
infinitely many nonzero solutions to the system.

Linear
Independence/Linear Dependence
This is not the first time that we’ve seen this topic. We also saw linear independence and linear
dependence back when we were looking at second
order differential equations. In that
section we were dealing with functions, but the concept is essentially the same
here. If we start with n vectors,
If we can find constants, c_{1},c_{2},…,c_{n} with at least two
nonzero such that
then we call the vectors linearly dependent. If the only constants that work in (4)
are c_{1}=0, c_{2}=0, …,
c_{n}=0 then we call the vectors linearly independent.
If we further make the assumption that each of the n vectors has n components, i.e. each
of the vectors look like,
we can get a very simple test for linear independence and
linear dependence. Note that this does
not have to be the case, but in all of our work we will be working with n vectors each of which has n components.
Fact
Given the n
vectors each with n components,
form the matrix,
So, the matrix X
is a matrix whose i^{th}
column is the i^{th}
vector, . Then,
 If X is nonsingular (i.e. det(X) is not zero) then the n
vectors are linearly independent, and
 if X is singular (i.e. det(X) = 0) then the n
vectors are linearly dependent and the constants that make (4)
true can be found by solving the system
where is a vector containing the constants
in (4).

Example 6 Determine
if the following set of vectors are linearly independent or linearly
dependent. If they are linearly
dependent find the relationship between them.
Solution
So, the first thing to do is to form X and compute its determinant.
This matrix is non singular and so the vectors are
linearly independent.

Example 7 Determine
if the following set of vectors are linearly independent or linearly
dependent. If they are linearly
dependent find the relationship between them.
Solution
As with the last example first form X and compute its determinant.
So, these vectors are linearly dependent. We now need to find the relationship
between the vectors. This means that
we need to find constants that will make (4)
true.
So we need to solve the system
Here is the augmented matrix and the solution work for
this system.
Now, we would like actual values for the constants so, if
use we get the following solution ,
,
and . The relationship is then.

Calculus with
Matrices
There really isn’t a whole lot to this other than to just
make sure that we can deal with calculus with matrices.
First, to this point we’ve only looked at matrices with
numbers as entries, but the entries in a matrix can be functions as well. So we can look at matrices in the following
form,
Now we can talk about differentiating and integrating a
matrix of this form. To differentiate or
integrate a matrix of this form all we do is differentiate or integrate the
individual entries.
So when we run across this kind of thing don’t get excited
about it. Just differentiate or
integrate as we normally would.
In this section we saw a very condensed set of topics from
linear algebra. When we get back to
differential equations many of these topics will show up occasionally and you
will at least need to know what the words mean.
The main topic from linear algebra that you must know
however if you are going to be able to solve systems of differential equations
is the topic of the next section.