Show Mobile Notice Show All Notes Hide All Notes

Section 5.2 : Review : Matrices & Vectors

This section is intended to be a catch all for many of the basic concepts that are used occasionally in working with systems of differential equations. There will not be a lot of details in this section, nor will we be working large numbers of examples. Also, in many cases we will not be looking at the general case since we won’t need the general cases in our differential equations work.

Let’s start with some of the basic notation for matrices. An \(n \times m\) (this is often called the size or dimension of the matrix) matrix is a matrix with \(n\) rows and \(m\) columns and the entry in the \(i^{\text{th}}\) row and \(j^{\text{th}}\) column is denoted by \(a_{ij}\). A short hand method of writing a general \(n \times m\) matrix is the following.

\[A = {\left( {\begin{array}{*{20}{r}}{{a_{11}}}&{{a_{12}}}& \cdots &{{a_{1m}}}\\{{a_{21}}}&{{a_{22}}}& \cdots &{{a_{2m}}}\\ \vdots & \vdots &{}& \vdots \\{{a_{n1}}}&{{a_{n2}}}& \cdots &{{a_{nm}}}\end{array}} \right)_{n \times m}} = {\left( {{a_{ij}}} \right)_{n \times m}}\]

The size or dimension of a matrix is subscripted as shown if required. If it’s not required or clear from the problem the subscripted size is often dropped from the matrix.

Special Matrices

There are a few “special” matrices out there that we may use on occasion. The first special matrix is the square matrix. A square matrix is any matrix whose size (or dimension) is \(n \times n\). In other words, it has the same number of rows as columns. In a square matrix the diagonal that starts in the upper left and ends in the lower right is often called the main diagonal.

The next two special matrices that we want to look at are the zero matrix and the identity matrix. The zero matrix, denoted \(0_{n \times m}\), is a matrix all of whose entries are zeroes. The identity matrix is a square \(n \times n\) matrix, denoted \(I_{n}\), whose main diagonals are all 1’s and all the other elements are zero. Here are the general zero and identity matrices.

\[{0_{n \times m}} = {\left( {\begin{array}{*{20}{r}}0&0& \cdots &0\\ \vdots & \vdots &{}& \vdots \\0&0& \cdots &0\end{array}} \right)_{n \times m}}\hspace{0.25in}\hspace{0.25in}{I_n} = {\left( {\begin{array}{*{20}{r}}1&0& \cdots &0\\0&1& \cdots &0\\ \vdots & \vdots & \ddots & \vdots \\0&0& \cdots &1\end{array}} \right)_{n \times n}}\]

In matrix arithmetic these two matrices will act in matrix work like zero and one act in the real number system.

The last two special matrices that we’ll look at here are the column matrix and the row matrix. These are matrices that consist of a single column or a single row. In general, they are,

\[x = {\left( {\begin{array}{*{20}{r}}{{x_1}}\\{{x_2}}\\ \vdots \\{{x_n}}\end{array}} \right)_{n \times 1}}\hspace{0.25in}\hspace{0.25in}\hspace{0.25in}y = {\left( {\begin{array}{*{20}{r}}{{y_1}}&{{y_2}}& \cdots &{{y_m}}\end{array}} \right)_{1 \times m}}\]

We will often refer to these as vectors.

Arithmetic

We next need to take a look at arithmetic involving matrices. We’ll start with addition and subtraction of two matrices. So, suppose that we have two \(n \times m\) matrices, \(A\) and \(B\). The sum (or difference) of these two matrices is then,

\[{A_{n \times m}} \pm {B_{n \times m}} = {\left( {{a_{ij}}} \right)_{n \times m}} \pm {\left( {{b_{ij}}} \right)_{n \times m}} = {\left( {{a_{ij}} \pm {b_{ij}}} \right)_{n \times m}}\]

The sum or difference of two matrices of the same size is a new matrix of identical size whose entries are the sum or difference of the corresponding entries from the original two matrices. Note that we can’t add or subtract entries with different sizes.

Next, let’s look at scalar multiplication. In scalar multiplication we are going to multiply a matrix \(A\) by a constant (sometimes called a scalar) \(\alpha \). In this case we get a new matrix whose entries have all been multiplied by the constant, \(\alpha \).

\[\alpha {A_{n \times m}} = \alpha {\left( {{a_{ij}}} \right)_{n \times m}} = {\left( {\alpha \,{a_{ij}}} \right)_{n \times m}}\]

The final matrix operation that we’ll take a look at is matrix multiplication. Here we will start with two matrices, \(A_{n \times p}\) and \(B_{p \times m}\). Note that \(A\) must have the same number of columns as \(B\) has rows. If this isn’t true, then we can’t perform the multiplication. If it is true, then we can perform the following multiplication.

\[{A_{n \times p}}{B_{p \times m}} = {\left( {{c_{ij}}} \right)_{n \times m}}\]

The new matrix will have size \(n \times m\) and the entry in the \(i^{\text{th}}\) row and \(j^{\text{th}}\) column, \(c_{ij}\), is found by multiplying row \(i\) of matrix \(A\) by column \(j\) of matrix \(B\). This doesn’t always make sense in words so let’s look at an example.

In this last example notice that we could not have done the product BA since the number of columns of \(B\) does not match the number of row of \(A\). It is important to note that just because we can compute \(AB\) doesn’t mean that we can compute \(BA\). Likewise, even if we can compute both \(AB\) and \(BA\) they may or may not be the same matrix.

Determinant

The next topic that we need to take a look at is the determinant of a matrix. The determinant is actually a function that takes a square matrix and converts it into a number. The actual formula for the function is somewhat complex and definitely beyond the scope of this review.

The main method for computing determinants of any square matrix is called the method of cofactors. Since we are going to be dealing almost exclusively with \(2 \times 2\) matrices and the occasional \(3 \times 3\) matrix we won’t go into the method here. We can give simple formulas for each of these cases. The standard notation for the determinant of the matrix \(A\) is.

\[\det \left( A \right) = \left| A \right|\]

Here are the formulas for the determinant of \(2 \times 2\) and \(3 \times 3\) matrices.

\[\left| {\begin{array}{*{20}{r}}a&c\\b&d\end{array}} \right| = ad - cb\] \[\left| {\begin{array}{*{20}{r}}{{a_{11}}}&{{a_{12}}}&{{a_{13}}}\\{{a_{21}}}&{{a_{22}}}&{{a_{23}}}\\{{a_{31}}}&{{a_{32}}}&{{a_{33}}}\end{array}} \right| = {a_{11}}\left| {\begin{array}{*{20}{r}}{{a_{22}}}&{{a_{23}}}\\{{a_{32}}}&{{a_{33}}}\end{array}} \right| - {a_{12}}\left| {\begin{array}{*{20}{r}}{{a_{21}}}&{{a_{23}}}\\{{a_{31}}}&{{a_{33}}}\end{array}} \right| + {a_{13}}\left| {\begin{array}{*{20}{r}}{{a_{21}}}&{{a_{22}}}\\{{a_{31}}}&{{a_{32}}}\end{array}} \right|\]

You can either use the formula or the short cut to get the determinant of a \(3 \times 3\).

If the determinant of a matrix is zero we call that matrix singular and if the determinant of a matrix isn’t zero we call the matrix nonsingular. The \(2 \times 2\) matrix in the above example was singular while the \(3 \times 3\) matrix is nonsingular.

Matrix Inverse

Next, we need to take a look at the inverse of a matrix. Given a square matrix, \(A\), of size n x \(n\) if we can find another matrix of the same size, \(B\) such that,

\[AB = BA = {I_n}\]

then we call \(B\) the inverse of \(A\) and denote it by \(B=A^{-1}\).

Computing the inverse of a matrix, \(A\), is fairly simple. First, we form a new matrix,

\[\left( {A\,\,\,{I_n}} \right)\]

and then use the row operations from the previous section and try to convert this matrix into the form,

\[\left( {{I_n}\,\,\,B} \right)\]

If we can then \(B\) is the inverse of \(A\). If we can’t then there is no inverse of the matrix \(A\).

So, there was an example in which the inverse did exist. Let’s take a look at an example in which the inverse doesn’t exist.

We will leave off this discussion of inverses with the following fact.

Fact

I’ll leave it to you to verify this fact for the previous two examples.

Systems of Equations Revisited

We need to do a quick revisit of systems of equations. Let’s start with a general system of equations.

\[\begin{equation}\begin{aligned}{a_{11}}{x_1} + {a_{12}}{x_2} + \cdots + {a_{1n}}{x_n} & = {b_1}\\ {a_{21}}{x_1} + {a_{22}}{x_2} + \cdots + {a_{2n}}{x_n} & = {b_2}\\ \vdots \hspace{0.8in} & \\ {a_{n1}}{x_1} + {a_{n2}}{x_2} + \cdots + {a_{nn}}{x_n} & = {b_n}\end{aligned}\label{eq:eq1}\end{equation}\]

Now, covert each side into a vector to get,

\[\left( {\begin{array}{*{20}{r}}{{a_{11}}{x_1} + {a_{12}}{x_2} + \cdots + {a_{1n}}{x_n}}\\{{a_{21}}{x_1} + {a_{22}}{x_2} + \cdots + {a_{2n}}{x_n}}\\ \vdots \\{{a_{n1}}{x_1} + {a_{n2}}{x_2} + \cdots + {a_{nn}}{x_n}}\end{array}} \right) = \left( {\begin{array}{*{20}{r}}{{b_1}}\\{{b_2}}\\ \vdots \\{{b_n}}\end{array}} \right)\]

The left side of this equation can be thought of as a matrix multiplication.

\[\left( {\begin{array}{*{20}{r}}{{a_{11}}}&{{a_{12}}}& \cdots &{{a_{1n}}}\\{{a_{21}}}&{{a_{22}}}& \cdots &{{a_{2n}}}\\ \vdots & \vdots & \ddots & \vdots \\{{a_{n1}}}&{{a_{n2}}}& \cdots &{{a_{nn}}}\end{array}} \right)\left( {\begin{array}{*{20}{r}}{{x_1}}\\{{x_2}}\\ \vdots \\{{x_n}}\end{array}} \right) = \left( {\begin{array}{*{20}{r}}{{b_1}}\\{{b_2}}\\ \vdots \\{{b_n}}\end{array}} \right)\]

Simplifying up the notation a little gives,

\[\begin{equation}A\vec x = \vec b \label{eq:eq2}\end{equation}\]

where, \(\vec x\) is a vector whose components are the unknowns in the original system of equations. We call \(\eqref{eq:eq2}\) the matrix form of the system of equations \(\eqref{eq:eq1}\) and solving \(\eqref{eq:eq2}\) is equivalent to solving \(\eqref{eq:eq1}\). The solving process is identical. The augmented matrix for \(\eqref{eq:eq2}\) is

\[\left( {A\,\,\,\vec b} \right)\]

Once we have the augmented matrix we proceed as we did with a system that hasn’t been written in matrix form.

We also have the following fact about solutions to \(\eqref{eq:eq2}\).

Fact

In fact, we can go a little farther now. Since we are assuming that we’ve got the same number of equations as unknowns the matrix \(A\) in \(\eqref{eq:eq2}\) is a square matrix and so we can compute its determinant. This gives the following fact.

Fact

The matrix form of a homogeneous system is

\[\begin{equation}A\vec x = \vec 0 \label{eq:eq3}\end{equation}\]

where \(\vec 0\) is the vector of all zeroes. In the homogeneous system we are guaranteed to have a solution, \(\vec x = \vec 0\). The fact above for homogeneous systems is then,

Fact

Linear Independence/Linear Dependence

This is not the first time that we’ve seen this topic. We also saw linear independence and linear dependence back when we were looking at second order differential equations. In that section we were dealing with functions, but the concept is essentially the same here. If we start with \(n\) vectors,

\[{\vec x_1},\,\,{\vec x_2},\,\, \ldots ,\,\,{\vec x_n}\]

If we can find constants, \(c_{1}\), \(c_{2}\), …, \(c_{n}\) with at least two nonzero such that

\[\begin{equation}{c_1}{\vec x_1} + {c_2}{\vec x_2} + \, \ldots + {c_n}{\vec x_n} = \vec 0 \label{eq:eq4}\end{equation}\]

then we call the vectors linearly dependent. If the only constants that work in \(\eqref{eq:eq4}\) are \(c_{1}=0\), \(c_{2}\)=0, …, \(c_{n}=0\) then we call the vectors linearly independent.

If we further make the assumption that each of the \(n\) vectors has \(n\) components, i.e. each of the vectors look like,

\[\vec x = \left( {\begin{array}{*{20}{r}}{{x_1}}\\{{x_2}}\\ \vdots \\{{x_n}}\end{array}} \right)\]

we can get a very simple test for linear independence and linear dependence. Note that this does not have to be the case, but in all of our work we will be working with \(n\) vectors each of which has \(n\) components.

Fact

Calculus with Matrices

There really isn’t a whole lot to this other than to just make sure that we can deal with calculus with matrices.

First, to this point we’ve only looked at matrices with numbers as entries, but the entries in a matrix can be functions as well. So, we can look at matrices in the following form,

\[A\left( t \right) = \left( {\begin{array}{*{20}{r}}{{a_{11}}\left( t \right)}&{{a_{12}}\left( t \right)}& \cdots &{{a_{1n}}\left( t \right)}\\{{a_{21}}\left( t \right)}&{{a_{22}}\left( t \right)}& \cdots &{{a_{2n}}\left( t \right)}\\ \vdots & \vdots &{}& \vdots \\{{a_{m1}}\left( t \right)}&{{a_{m2}}\left( t \right)}& \cdots &{{a_{mn}}\left( t \right)}\end{array}} \right)\]

Now we can talk about differentiating and integrating a matrix of this form. To differentiate or integrate a matrix of this form all we do is differentiate or integrate the individual entries.

\[A'\left( t \right) = \left( {\begin{array}{*{20}{r}}{{{a'}_{11}}\left( t \right)}&{{{a'}_{12}}\left( t \right)}& \cdots &{{{a'}_{1n}}\left( t \right)}\\{{{a'}_{21}}\left( t \right)}&{{{a'}_{22}}\left( t \right)}& \cdots &{{{a'}_{2n}}\left( t \right)}\\ \vdots & \vdots &{}& \vdots \\{{{a'}_{m1}}\left( t \right)}&{{{a'}_{m2}}\left( t \right)}& \cdots &{{{a'}_{mn}}\left( t \right)}\end{array}} \right)\] \[\int{{A\left( t \right)\,dt}} = \left( {\begin{array}{*{20}{r}}{\int{{{a_{11}}\left( t \right)\,dt}}}&{\int{{{a_{12}}\left( t \right)\,dt}}}& \cdots &{\int{{{a_{1n}}\left( t \right)\,dt}}}\\{\int{{{a_{21}}\left( t \right)\,dt}}}&{\int{{{a_{22}}\left( t \right)\,dt}}}& \cdots &{\int{{{a_{2n}}\left( t \right)\,dt}}}\\ \vdots & \vdots &{}& \vdots \\{\int{{{a_{m1}}\left( t \right)\,dt}}}&{\int{{{a_{m2}}\left( t \right)\,dt}}}& \cdots &{\int{{{a_{mn}}\left( t \right)\,dt}}}\end{array}} \right)\]

So, when we run across this kind of thing don’t get excited about it. Just differentiate or integrate as we normally would.

In this section we saw a very condensed set of topics from linear algebra. When we get back to differential equations many of these topics will show up occasionally and you will at least need to know what the words mean.

The main topic from linear algebra that you must know however if you are going to be able to solve systems of differential equations is the topic of the next section.