PROFESSOR: Hi, everyone.
Welcome back.
So today, I'd like to talk about
positive definite matrices.
And specifically, we're going
to analyze several properties
of positive definite matrices.
And specifically,
we're going to look
at why each one of these
following statements is true.
So first off, why every positive
definite matrix is invertible.
Why the only positive
definite projection matrix
is the identity matrix.
If D is a diagonal matrix
with positive entries,
show that it must also
be positive definite.
And then lastly, if S is
a symmetric matrix where
the determinant S
is bigger than 0,
show why this might
not necessarily
imply that it's
positive definite.
So I'll let you think
about these for a moment.
And I'll come back in a second.
Hi, everyone.
Welcome back.
OK.
So let's take a look
at part A. So part A
is asking why every positive
definite matrix is invertible.
Well, let's just recall
that if A is a matrix
and if A is invertible,
then this necessarily
implies that the determinant
of A is non-zero.
And I'm going to
just write out det
A as the product of the
eigenvalues of A. So lambda_1
to lambda_n are the
eigenvalues of A.
OK.
In addition, if A is
positive definite,
what does this say about
the eigenvalues of A?
Well, it says that each
eigenvalue of A, lambda_1,
lambda_2, dot, dot, dot, to
lambda_n, each one of them
must be bigger than 0.
So this statement that
each eigenvalue of A
is bigger than 0 is
completely equivalent
to A being positive definite
for symmetric matrices A.
So if I have a whole bunch of
eigenvalues and each of them
are bigger than 0, what
does this say about det A?
Well, I can take the product
of all these eigenvalues.
And of course, the product of a
whole bunch of positive numbers
must also be positive.
So if the quantity is positive,
then it certainly cannot equal
0.
So this proves that det
A is not equal to 0.
Hence, A must be invertible.
OK.
So for part B,
we're asked to show
that the only positive
definite projection
matrix is the identity matrix.
So again, how do we
tackle this problem?
We're going to look
at the eigenvalues.
So remember, if P
is a projection,
what does it say about
the eigenvalues of P?
Well, it says that
the eigenvalues of P
are either 0 or 1.
So this is point one.
Point two, if P is
a positive definite,
what does that say about
the eigenvalues of P?
Well, as I've noted before,
it means that the eigenvalues
are bigger than 0.
So if P is a projection
and it's positive definite,
the only possible
eigenvalues that
are both 0 and 1 and
bigger than 0 are 1.
So the conclusion is
that the eigenvalues of P
must all equal 1.
So which matrix has eigenvalues
1 and is also symmetric?
Well, the only matrix that
satisfies this property
is the identity matrix.
Now you might ask, how do
you actually show that?
Well, you could
argue as follows.
If P is diagonalizable--
and every symmetric matrix
is diagonalizable, so
I'm not making this up.
So if P is
diagonalizable, then you
can always write P
as some matrix, U,
times a diagonal matrix--
and we know in this case,
the diagonal matrix
has eigenvalues 1,
so it's actually the identity
matrix-- times the inverse
of the eigenvector matrix.
But of course, this is
just U times U inverse,
which then gives me the
identity at the end.
So U times the identity
times U inverse.
This is just U times U inverse.
And of course, U and U
inverse collapse back down
to the identity.
So this shows you that the only
matrix that has eigenvalues
of 1 is the identity matrix.
So that's just to cross all
the T's and dot all the I's.
OK.
For part C, we're given
D as a diagonal matrix
with positive entries
on the diagonal.
Now we have to show that
it's positive definite.
OK.
So let me write D as follows.
I'll just write it like this.
I'm going to use a compact
notation, which is sometimes
seen: diagonal d_1, d_2, d_n.
So D is a diagonal matrix whose
elements along the diagonal
are d_1, d_2, dot,
dot, dot, to d_n.
Now what does it mean for a
matrix to be positive definite?
Well, it means that for
any x, for any vector x,
I have to look at the
product x transpose D*x.
And I have to show that
it's bigger than 0.
And I should
qualify this and say
that the vector we're looking
at is x not equal to 0.
So we're now looking
at the zero vector.
But for D to be
positive definite,
we have to show that x
transpose D*x is bigger than 0.
This is just one way to show
that it's positive definite.
It's not the only way.
So if I write x out using
components, x_1, x_2, dot, dot,
dot, to x_n-- I'll
write it like this--
then you can work out the
quantity x transpose D*x.
And we see that we
get a sum of squares.
We get d_1 times x_1 squared
plus d_2 x_2 squared plus dot,
dot, dot plus d_n x_n squared.
Now by definition, each
coefficient is positive.
A sum of a square is positive.
so-- sorry.
A product of a positive number
with a square is positive.
And then of course, a
sum of positive numbers
is going to be positive.
So this means the whole
thing is positive.
Now there's other
more efficient ways
of getting at this using
other tricks we know.
For example, if we're
given a diagonal matrix,
we know its eigenvalues
are already d_1,
d_2, dot, dot, dot, to d_n.
And we know that a matrix
with positive eigenvalues
is already positive definite.
But this is kind of
starting from the base
to show that it's
positive definite.
And now lastly,
let's look at part D.
So S is a symmetric matrix
with det S bigger than 0.
Show that S might
not necessarily
be positive definite.
So there's lots of
counterexamples.
I only need to construct one.
So I'm looking at S, which
is a symmetric matrix.
So I'm just going to throw
in some numbers on some
off diagonals.
So I'll just pick one.
And now I need to pick some
numbers along the diagonal.
And the easiest way
to do it is just
to pick two negative
numbers on the diagonal.
Because if there's a negative
number on a diagonal,
then we know S can't
be positive definite.
And I'll say more
about that in a second.
So I can just pick
negative 2 and negative 3.
So let's quickly
check what det S is.
Well, it's negative 2
times negative 3 minus 1.
So this gives me 6
minus 1, which is 5.
So by construction, det S
is positive, which is good.
And then, as I mentioned
before, if there's
negative elements along
the diagonal of the matrix,
that matrix can't be
positive definite.
Well, why is that?
Well, suppose I
wanted to take a look
at this upper left
component, negative 3,
and it's negative, how do
I show that that implies
S is not positive definite?
Well, what I can do is I can
look at the product x transpose
S*x.
And what I do is I look at it.
I just take one value of x.
So we know that this has to be
positive for every value of x.
So I can pick any value I want.
So I can take x, say,
[1, 0] transpose.
And when I do this, we end up
getting that x transpose S*x is
equal to negative 3.
So notice how by taking 1
in the first entry and 0
on the second entry, that picks
out the upper left corner,
negative 3.
If I were to take
0 here and 1 here,
it would pick out the
negative 2 entry in S.
So by picking 1 in
entry i of a vector x,
and then computing this
product x transpose S*x,
I pick out the i-th
element along the diagonal.
And since that
element is negative,
this shows me that along
the direction [1, 0],
the product x transpose
S*x is also negative.
And hence, S can't possibly
be positive definite.
OK.
So just to summarize,
we've taken a look
at a couple matrices and a
couple different properties
of positive definite matrices.
And notably, we've
used the eigenvalues
to get a handle of the
positive definite matrices.
And I hope these provide
some useful tricks.
And I'll see you next time.
