Column space of a matrix

WEBVTT
Kind: captions
Language: en

00:00:00.470 --> 00:00:03.490
We spent a good deal of time on
the idea of a null space.

00:00:03.490 --> 00:00:05.720
What I'm going to do in this
video is introduce you to a

00:00:05.720 --> 00:00:09.090
new type of space that can be
defined around a matrix, it's

00:00:09.090 --> 00:00:12.930
called a column space.

00:00:12.930 --> 00:00:15.770
And you could probably guess
what it means just based on

00:00:15.770 --> 00:00:16.760
what it's called.

00:00:16.760 --> 00:00:20.200
But let's say I have
some matrix A.

00:00:20.200 --> 00:00:23.600
Let's say it's an
m by n matrix.

00:00:23.600 --> 00:00:26.270
So I can write my matrix A and
we've seen this multiple

00:00:26.270 --> 00:00:30.300
times, I can write it as a
collection of columns vectors.

00:00:30.300 --> 00:00:34.430
So this first one, second one,
and I'll have n of them.

00:00:34.430 --> 00:00:36.110
How do I know that
I have n of them?

00:00:36.110 --> 00:00:39.330
Because I have n columns.

00:00:39.330 --> 00:00:41.070
And each of these column
vectors, we're going to have

00:00:41.070 --> 00:00:42.500
how many components?

00:00:42.500 --> 00:00:47.650
So v1, v2, all the way to vn.

00:00:47.650 --> 00:00:50.840
This matrix has m rows.

00:00:50.840 --> 00:00:52.660
So each of these guys are going
to have m components.

00:00:52.660 --> 00:00:56.380
So they're all members of Rm.

00:00:56.380 --> 00:01:01.250
So the column space is defined
as all of the possible linear

00:01:01.250 --> 00:01:03.860
combinations of these
columns vectors.

00:01:03.860 --> 00:01:09.270
So the column space of A, this
is my matrix A, the column

00:01:09.270 --> 00:01:11.780
space of that is all the linear
combinations of these

00:01:11.780 --> 00:01:12.790
column vectors.

00:01:12.790 --> 00:01:13.940
What's all of the linear

00:01:13.940 --> 00:01:15.780
combinations of a set of vectors?

00:01:15.780 --> 00:01:21.090
It's the span of
those vectors.

00:01:21.090 --> 00:01:25.430
So it's the span of vector
1, vector 2, all the

00:01:25.430 --> 00:01:28.360
way to vector n.

00:01:28.360 --> 00:01:30.210
And we've done it before
when we first talked

00:01:30.210 --> 00:01:31.240
about span and subspaces.

00:01:31.240 --> 00:01:33.980
But it's pretty easy to show
that the span of any set of

00:01:33.980 --> 00:01:35.980
vectors is a legitimate
subspace.

00:01:35.980 --> 00:01:38.180
It definitely contains
the 0 vector.

00:01:38.180 --> 00:01:41.760
If you multiply all of these
guys by 0, which is a valid

00:01:41.760 --> 00:01:44.250
linear combination added up,
you'll see that it contains

00:01:44.250 --> 00:01:45.430
the 0 vector.

00:01:45.430 --> 00:01:54.780
If, let's say that I have some
vector a that is a member of

00:01:54.780 --> 00:01:56.560
the column space of a.

00:01:56.560 --> 00:01:58.680
That means it can be represented
as some linear

00:01:58.680 --> 00:01:59.910
combination.

00:01:59.910 --> 00:02:05.710
So a is equal to c1 times vector
1, plus c2 times vector

00:02:05.710 --> 00:02:10.320
2, all the way to Cn
times vector n.

00:02:10.320 --> 00:02:13.360
Now, the question is, is this
closed under multiplication?

00:02:13.360 --> 00:02:18.200
If I multiply a times some new--
let me say I multiply it

00:02:18.200 --> 00:02:22.880
times some scale or s, I'm just
picking a random letter--

00:02:22.880 --> 00:02:27.090
so s times a, is this
in my span?

00:02:27.090 --> 00:02:35.240
Well s times a would be equal
to s c1 v1 plus s c2 v2, all

00:02:35.240 --> 00:02:40.900
the way to s Cn Vn Which is
once again just a linear

00:02:40.900 --> 00:02:43.150
combination of these
column vectors.

00:02:43.150 --> 00:02:47.880
So this Sa, would clearly
be a member of the

00:02:47.880 --> 00:02:49.950
column space of a.

00:02:49.950 --> 00:02:52.650
And then finally, to make sure
it's a valid subspace-- and

00:02:52.650 --> 00:02:54.260
this actually doesn't apply just
to column space, so this

00:02:54.260 --> 00:02:55.470
applies to any span.

00:02:55.470 --> 00:02:57.700
This is actually a review of
what we've done the past. We

00:02:57.700 --> 00:03:00.000
just have to make sure it's
closed under addition.

00:03:00.000 --> 00:03:02.690
So let's say a is a member
of our column space.

00:03:02.690 --> 00:03:06.510
Let's say b is also a member
of our column space, or our

00:03:06.510 --> 00:03:08.950
span of all these
column vectors.

00:03:08.950 --> 00:03:18.150
Then b could be written as b1
times v1, plus b2 times v2,

00:03:18.150 --> 00:03:21.470
all the way to Bn times Vn.

00:03:21.470 --> 00:03:29.830
And my question is, is a plus b
a member of our span, of our

00:03:29.830 --> 00:03:33.150
column space, the span
of these vectors?

00:03:33.150 --> 00:03:41.670
Well sure, what's a plus b? a
plus b is equal to c1 plus b1

00:03:41.670 --> 00:03:50.420
times v1, plus c2 plus
v2 times v2.

00:03:50.420 --> 00:03:52.310
I'm just literally adding
this term to that

00:03:52.310 --> 00:03:53.610
term, to get that term.

00:03:53.610 --> 00:03:55.945
This term to this term
to get this term.

00:03:55.945 --> 00:04:01.710
And then it goes all the way
to Bn and plus Cn times Vn.

00:04:01.710 --> 00:04:03.710
Which is clearly just
another linear

00:04:03.710 --> 00:04:05.110
combination of these guys.

00:04:05.110 --> 00:04:10.530
So this guy is definitely
within the span.

00:04:10.530 --> 00:04:12.870
It doesn't have to be
unique to a matrix.

00:04:12.870 --> 00:04:15.200
A matrix is just really just
a way of writing a

00:04:15.200 --> 00:04:16.140
set of column vectors.

00:04:16.140 --> 00:04:18.029
So this applies to any span.

00:04:18.029 --> 00:04:21.700
So this is clearly
a valid subspace.

00:04:21.700 --> 00:04:27.800
So the column space of a is
clearly a valid subspace.

00:04:27.800 --> 00:04:31.280
Let's think about other ways we
can interpret this notion

00:04:31.280 --> 00:04:32.930
of a column space.

00:04:32.930 --> 00:04:39.150
Let's think about it in terms of
the expression-- let me get

00:04:39.150 --> 00:04:43.900
a good color-- if I were to
multiply my-- let's think

00:04:43.900 --> 00:04:45.660
about this.

00:04:45.660 --> 00:04:50.470
Let's think about the set of all
the values of if I take my

00:04:50.470 --> 00:05:00.790
m by n matrix a and I multiply
it by any vector x, where x is

00:05:00.790 --> 00:05:04.550
a member of-- remember x has
to be a member of Rn.

00:05:04.550 --> 00:05:07.350
It has to have n components in
order for this multiplication

00:05:07.350 --> 00:05:09.070
to be well defined.

00:05:09.070 --> 00:05:12.770
So x has to be a member of Rn.

00:05:12.770 --> 00:05:15.260
Let's think about
what this means.

00:05:15.260 --> 00:05:19.180
This says, look, I can take any
member, any n component

00:05:19.180 --> 00:05:22.840
vector and multiply it by a,
and I care about all of the

00:05:22.840 --> 00:05:26.060
possible products that this
could equal, all the possible

00:05:26.060 --> 00:05:29.830
values of Ax, when I can
pick and choose any

00:05:29.830 --> 00:05:32.380
possible x from Rn.

00:05:32.380 --> 00:05:33.630
Let's think about
what that means.

00:05:36.490 --> 00:05:40.470
If I write a like that, and if
I write x like this-- let me

00:05:40.470 --> 00:05:45.180
write it a little bit better,
let me write x like this-- x1,

00:05:45.180 --> 00:05:48.680
x2, all the way to Xn.

00:05:48.680 --> 00:05:50.320
What is Ax?

00:05:50.320 --> 00:05:53.680
Well Ax could be rewritten as
x1-- and we've seen this

00:05:53.680 --> 00:06:09.640
before-- Ax is equal to x1 times
v1 plus x2 times v2, all

00:06:09.640 --> 00:06:17.950
the way to plus Xn times Vn.

00:06:17.950 --> 00:06:19.200
We've seen this multiple
times.

00:06:19.200 --> 00:06:20.980
This comes out of our
definition of

00:06:20.980 --> 00:06:23.190
matrix vector products.

00:06:23.190 --> 00:06:28.350
Now if Ax is equal to this,
and I'm essentially saying

00:06:28.350 --> 00:06:31.740
that I can pick any vector x in
Rn, I'm saying that I can

00:06:31.740 --> 00:06:35.590
pick all possible values of the
entries here, all possible

00:06:35.590 --> 00:06:38.400
real values and all possible
combinations of them.

00:06:38.400 --> 00:06:39.800
So what is this equal to?

00:06:39.800 --> 00:06:42.290
What is the set of
all possible?

00:06:42.290 --> 00:06:46.420
So I could rewrite this
statement here as the set of

00:06:46.420 --> 00:06:57.750
all possible x1 v1 plus x2 v2
all the way to Xn Vn, where

00:06:57.750 --> 00:07:03.300
x1, x2, all the way to Xn, are
a member of the real numbers.

00:07:03.300 --> 00:07:04.320
That's all I'm saying here.

00:07:04.320 --> 00:07:05.910
This statement is the
equivalent of this.

00:07:05.910 --> 00:07:09.590
When I say that the vector x can
be any member of Rn, I'm

00:07:09.590 --> 00:07:11.840
saying that its components
can be any

00:07:11.840 --> 00:07:13.340
members of the real numbers.

00:07:13.340 --> 00:07:16.690
So if I just take the set of all
of the, essentially, the

00:07:16.690 --> 00:07:19.320
combinations of these column
vectors where their real

00:07:19.320 --> 00:07:23.570
numbers, where their
coefficients, are members of

00:07:23.570 --> 00:07:24.270
the real numbers.

00:07:24.270 --> 00:07:25.010
What am I doing?

00:07:25.010 --> 00:07:28.070
This is all the possible linear
combinations of the

00:07:28.070 --> 00:07:29.740
column vectors of a.

00:07:29.740 --> 00:07:42.000
So this is equal to the span
v1 v2, all the way to Vn,

00:07:42.000 --> 00:07:46.920
which is the exact same thing
as the column space of A.

00:07:46.920 --> 00:07:49.930
So the column space of A, you
could say what are all of the

00:07:49.930 --> 00:07:53.170
possible vectors, or the set of
all vectors I can create by

00:07:53.170 --> 00:07:55.860
taking linear combinations of
these guys, or the span of

00:07:55.860 --> 00:07:56.770
these guys.

00:07:56.770 --> 00:08:00.300
Or you can view it as, what are
all of the possible values

00:08:00.300 --> 00:08:05.580
that Ax can take on if
x is a member of Rn?

00:08:05.580 --> 00:08:08.010
So let's think about
it this way.

00:08:08.010 --> 00:08:12.830
Let's say that I were to tell
you that I need to solve the

00:08:12.830 --> 00:08:20.340
equation Ax is equal to-- well
the convention is to write a b

00:08:20.340 --> 00:08:21.590
there-- but let me
put a special b

00:08:21.590 --> 00:08:24.560
there, let me put b1.

00:08:24.560 --> 00:08:26.340
Let's say that I need to
solve this equation

00:08:26.340 --> 00:08:28.000
Ax is equal to b1.

00:08:28.000 --> 00:08:30.070
And then I were to tell you--
let's say that I were to

00:08:30.070 --> 00:08:35.870
figure out the column space of
A-- and I say b1 is not a

00:08:35.870 --> 00:08:40.710
member of the column space of
A So what does that tell me?

00:08:40.710 --> 00:08:45.550
That tells me that this right
here can never take on the

00:08:45.550 --> 00:08:49.940
value b1 because all of the
values that this can take on

00:08:49.940 --> 00:08:51.900
is the column space of A.

00:08:51.900 --> 00:08:55.000
So if b1 is not in this, it
means that this cannot take on

00:08:55.000 --> 00:08:56.200
the value of b1.

00:08:56.200 --> 00:09:00.710
So this would imply that this
equation we're trying to set

00:09:00.710 --> 00:09:04.283
up, Ax is equal to b1,
has no solution.

00:09:06.850 --> 00:09:18.710
If it had a solution, so let's
say that Ax equals b2 has at

00:09:18.710 --> 00:09:19.960
least one solution.

00:09:25.820 --> 00:09:27.150
What does this mean?

00:09:27.150 --> 00:09:31.570
Well, that means that this, for
a particular x or maybe

00:09:31.570 --> 00:09:33.540
for many different x's,
you can definitely

00:09:33.540 --> 00:09:36.110
achieve this value.

00:09:36.110 --> 00:09:38.600
For there are some x's that when
you multiply it by a, you

00:09:38.600 --> 00:09:40.660
definitely are able
to get this value.

00:09:40.660 --> 00:09:46.020
So this implies that b2 is
definitely a member of the

00:09:46.020 --> 00:09:48.480
column space of A.

00:09:48.480 --> 00:09:51.000
Some of this stuff on some level
it's almost obvious.

00:09:51.000 --> 00:09:52.410
This comes out of
the definition

00:09:52.410 --> 00:09:53.280
of the column space.

00:09:53.280 --> 00:09:55.570
The column space is all of the
linear combinations of the

00:09:55.570 --> 00:09:58.740
column vectors, which another
interpretation is all of the

00:09:58.740 --> 00:10:01.260
values that Ax can take on.

00:10:01.260 --> 00:10:04.420
So if I try to set Ax to some
value that it can't take on,

00:10:04.420 --> 00:10:07.690
clearly I'm not going to
have some solution.

00:10:07.690 --> 00:10:12.120
If I am able to find a solution,
I am able to find

00:10:12.120 --> 00:10:16.590
some x value where Ax is equal
to b2, then b2 definitely is

00:10:16.590 --> 00:10:20.140
one of the values that
Ax can take on.

00:10:20.140 --> 00:10:21.580
Anyway, I think I'll
leave you there.

00:10:21.580 --> 00:10:24.830
Now that you have at least a
kind of abstract understanding

00:10:24.830 --> 00:10:26.330
of what a column space is.

00:10:26.330 --> 00:10:28.080
In the next couple of videos
I'm going to try to bring

00:10:28.080 --> 00:10:31.060
everything together of what we
know about column spaces, and

00:10:31.060 --> 00:10:34.970
null spaces, and whatever else
to kind of understand a matrix

00:10:34.970 --> 00:10:39.280
and a matrix vector product from
every possible direction.


Null space 3: Relation to linear independence

WEBVTT
Kind: captions
Language: en

00:00:00.560 --> 00:00:03.780
The neat thing about linear
algebra in general is some

00:00:03.780 --> 00:00:07.630
very seemingly simple concepts
can be interpreted in a bunch

00:00:07.630 --> 00:00:10.740
of different ways, and can be
shown to represent different

00:00:10.740 --> 00:00:12.540
ideas or different problems. And
that's what I'm going to

00:00:12.540 --> 00:00:13.260
do in this video.

00:00:13.260 --> 00:00:15.660
I'm going to explore the
nullspace, or even better, I'm

00:00:15.660 --> 00:00:18.990
going to explore the
relationship-- if I have some

00:00:18.990 --> 00:00:24.900
matrix A times some vector
x, and that that is

00:00:24.900 --> 00:00:26.560
equal to the 0 vector.

00:00:26.560 --> 00:00:28.890
And we, of course, saw on the
last few videos that the

00:00:28.890 --> 00:00:38.520
nullspace of A, is equal to all
of the vectors x in Rn.

00:00:38.520 --> 00:00:39.840
So this will have
n components.

00:00:39.840 --> 00:00:42.375
This would have to be
an m by n matrix.

00:00:42.375 --> 00:00:45.490
If this was an m by a matrix,
I'd say all of

00:00:45.490 --> 00:00:47.330
the vectors in Ra.

00:00:47.330 --> 00:00:50.800
So this number right here has to
be the same as that number

00:00:50.800 --> 00:00:54.530
in order for the matrix vector
multiplication to be valid.

00:00:54.530 --> 00:00:58.490
But the nullspace of A is all
of the vectors in Rn that

00:00:58.490 --> 00:01:00.110
satisfy this equation.

00:01:00.110 --> 00:01:03.280
Where if I take A and I multiply
it times any one of

00:01:03.280 --> 00:01:09.600
the vectors in the nullspace,
I should get the 0 vector.

00:01:09.600 --> 00:01:12.640
And this is going to have m
components, and we've seen

00:01:12.640 --> 00:01:15.680
that in previous-- this is going
to have the 0 vectors.

00:01:15.680 --> 00:01:17.270
I'll put it this way, the
0 vector's going to

00:01:17.270 --> 00:01:21.170
be a member of Rm.

00:01:21.170 --> 00:01:22.280
So that's what our
nullspace is.

00:01:22.280 --> 00:01:24.080
Let's explore it a little bit.

00:01:24.080 --> 00:01:26.930
We know already that our vector,
our matrix, can be

00:01:26.930 --> 00:01:28.250
rewritten like this.

00:01:28.250 --> 00:01:30.410
We could just write it as
a set of column vectors.

00:01:30.410 --> 00:01:34.410
I could say this right here,
that's v1, then I'm going to

00:01:34.410 --> 00:01:37.640
have v2, and I have n columns.

00:01:37.640 --> 00:01:41.980
So this last column right here
is going to be v sub n.

00:01:41.980 --> 00:01:46.210
So if I define my vectors this
way, that's the first vector,

00:01:46.210 --> 00:01:48.870
that's the second vector, than
I can rewrite my matrix A.

00:01:48.870 --> 00:01:53.780
I could say A is equal to just
a bunch of column vectors.

00:01:53.780 --> 00:02:00.120
v1, v2, all the way to vn.

00:02:00.120 --> 00:02:06.850
And multiplying this matrix
times a vector x, so times x,

00:02:06.850 --> 00:02:11.840
so times x1, x2, all
the way to xn.

00:02:11.840 --> 00:02:15.100
We've seen in the past on the
matrix vector product

00:02:15.100 --> 00:02:18.790
definition video that this can
be interpreted as, this has

00:02:18.790 --> 00:02:20.400
actually just coming straight
out of the definition.

00:02:20.400 --> 00:02:25.820
This is the same thing as x1
times vector 1, times the

00:02:25.820 --> 00:02:33.760
first column, plus x2 times the
second column, times that

00:02:33.760 --> 00:02:37.500
column, all the way to, and you
just keep adding them up,

00:02:37.500 --> 00:02:39.500
all the way to xn.

00:02:42.150 --> 00:02:43.490
Times the nth column.

00:02:43.490 --> 00:02:47.110
This just comes straight out
of our definition of matrix

00:02:47.110 --> 00:02:48.740
vector products.

00:02:48.740 --> 00:02:55.200
Now, if we're saying that Ax is
equal to 0, we're looking

00:02:55.200 --> 00:02:57.170
for the solution set to that.

00:02:57.170 --> 00:02:59.990
If we're looking for the
solution set to Ax is equal to

00:02:59.990 --> 00:03:04.080
0, then that means-- is equal
to the 0 vector, that that

00:03:04.080 --> 00:03:07.840
means that this sum, we're
trying to find a solution set

00:03:07.840 --> 00:03:11.020
of this sum is equaling 0.

00:03:11.020 --> 00:03:14.090
We want to figure out the x1's,
x2's, x3's, all the way

00:03:14.090 --> 00:03:19.130
to xn's, that make this
equal the 0 vector.

00:03:19.130 --> 00:03:20.090
What are we doing?

00:03:20.090 --> 00:03:22.040
We're taking linear
combinations

00:03:22.040 --> 00:03:23.860
of our column vectors.

00:03:23.860 --> 00:03:25.820
We're taking linear combinations
of our column

00:03:25.820 --> 00:03:29.120
vectors, and seeing if we can
take some linear combination

00:03:29.120 --> 00:03:31.180
and get it to the 0 vector.

00:03:31.180 --> 00:03:35.310
Now, this should start ringing
bells in your head.

00:03:35.310 --> 00:03:39.480
This little equation, or this
little expression right here,

00:03:39.480 --> 00:03:41.390
should start ringing bells.

00:03:41.390 --> 00:03:46.590
This was part of how we
defined what linear

00:03:46.590 --> 00:03:48.690
independence was.

00:03:48.690 --> 00:03:51.050
We said that if this was the
definition of linear

00:03:51.050 --> 00:03:52.730
independence, or we proved
this fell out of the

00:03:52.730 --> 00:03:54.920
definition of linear
independence, and if I have a

00:03:54.920 --> 00:04:00.720
bunch of vectors, v1, v2, all
the way to vn, we say that

00:04:00.720 --> 00:04:05.325
they are linearly independent.

00:04:09.040 --> 00:04:11.500
There's kind of the
non-mathematical way of

00:04:11.500 --> 00:04:14.360
describing it, I guess this is
mathematical as well, is that

00:04:14.360 --> 00:04:17.200
look, none of those vectors
can be represented as a

00:04:17.200 --> 00:04:18.540
combination of the other ones.

00:04:18.540 --> 00:04:21.440
And then we show that that means
that the only solution

00:04:21.440 --> 00:04:26.520
to this equation would be
that x1, x2, all of the

00:04:26.520 --> 00:04:30.240
coefficients on this, has
to be equal to 0.

00:04:30.240 --> 00:04:32.450
That this is the
only solution.

00:04:32.450 --> 00:04:37.490
Linear independence means that
this is the only solution to

00:04:37.490 --> 00:04:40.300
this equation right now.

00:04:40.300 --> 00:04:43.540
If the only way that you get
the 0 vector, by taking

00:04:43.540 --> 00:04:46.170
combination of all of these
common vectors, the only way

00:04:46.170 --> 00:04:50.400
to do that is to have all
of these guys equal 0.

00:04:50.400 --> 00:04:52.040
Then you are linearly
independent.

00:04:52.040 --> 00:04:55.370
Likewise, if v1, v2, all the
way to vn are linearly

00:04:55.370 --> 00:04:57.750
independent, then the only
solution to this is for these

00:04:57.750 --> 00:04:59.930
coefficients to be 0.

00:04:59.930 --> 00:05:02.920
And we saw that in our video
on linear independence.

00:05:02.920 --> 00:05:04.590
Now, if all of these
coefficients are 0,

00:05:04.590 --> 00:05:05.570
what does that mean?

00:05:05.570 --> 00:05:14.670
That means that our vector
x is the 0 vector,

00:05:14.670 --> 00:05:16.080
and only the 0 vector.

00:05:16.080 --> 00:05:18.050
That's the only solution.

00:05:18.050 --> 00:05:20.220
So we have something
interesting here.

00:05:20.220 --> 00:05:25.890
If our column vectors are
linearly independent, if v1,

00:05:25.890 --> 00:05:35.960
v2, all the way to vn, are
linearly independent, then

00:05:35.960 --> 00:05:50.370
that means that the only
solution to Ax equals 0, is

00:05:50.370 --> 00:05:52.670
that x has to be equal
to 0 vector.

00:05:52.670 --> 00:05:56.380
Or put another way, the solution
set of this equation,

00:05:56.380 --> 00:05:58.410
which is really just a
nullspace, the nullspace is

00:05:58.410 --> 00:06:01.770
all of the x's that satisfy
this equation.

00:06:01.770 --> 00:06:08.480
So that the nullspace of A has
to only contain the 0 vector.

00:06:08.480 --> 00:06:09.690
So that's an interesting
result.

00:06:09.690 --> 00:06:14.300
If we're linearly independent,
then the nullspace of A only

00:06:14.300 --> 00:06:16.010
contains the 0 vector.

00:06:16.010 --> 00:06:19.870
Which is another way of saying
that-- let me write this--

00:06:19.870 --> 00:06:23.280
well, I already wrote it down,
that x1, x2, all of them, have

00:06:23.280 --> 00:06:24.810
to be equal to 0.

00:06:24.810 --> 00:06:28.610
Now if I were to multiply this
equation out and get it into

00:06:28.610 --> 00:06:31.530
reduced row echelon form,
what does that mean?

00:06:31.530 --> 00:06:37.090
We saw in a previous video that
the nullspace of A is

00:06:37.090 --> 00:06:44.260
equal to the nullspace of the
reduced row echelon form of A.

00:06:44.260 --> 00:06:47.110
And that's-- the nullspace of
A is 0, because its column

00:06:47.110 --> 00:06:49.500
vectors are each linearly
independent, and that means

00:06:49.500 --> 00:06:52.960
that the nullspace of the
reduced row echelon form of A

00:06:52.960 --> 00:06:57.030
must also equal the 0 vector.

00:06:57.030 --> 00:07:00.150
And that means that if I take
the reduced row echelon form

00:07:00.150 --> 00:07:05.050
of A, times-- maybe I'm being
a little redundant-- the

00:07:05.050 --> 00:07:09.390
reduced row echelon form of A,
and I multiply that times x,

00:07:09.390 --> 00:07:13.900
or I want to solve this
equation, the only solution

00:07:13.900 --> 00:07:19.530
right here is x is equal
to the 0 vector.

00:07:19.530 --> 00:07:21.790
And if you think about what
that means, if this is the

00:07:21.790 --> 00:07:25.840
only solution, that means that
this reduced row echelon form

00:07:25.840 --> 00:07:27.530
has no free variables.

00:07:27.530 --> 00:07:31.390
It literally would just have
to look like this.

00:07:31.390 --> 00:07:38.920
So this is x, x1, x2, all the
way to xn, the reduced row

00:07:38.920 --> 00:07:42.840
echelon form of A, in order
for this to have a unique

00:07:42.840 --> 00:07:45.640
solution, and that unique
solution being 0, the reduced

00:07:45.640 --> 00:07:47.640
echelon form is going to
have to look like this.

00:07:51.840 --> 00:07:55.200
1 times x1 plus 0 times all
the other ones, so you're

00:07:55.200 --> 00:07:58.530
going to have just a bunch of
n0's, and you're going to have

00:07:58.530 --> 00:08:02.150
1 times x2, plus 0's times
everything else.

00:08:02.150 --> 00:08:05.170
And those 1's are going to go
all the way down the diagonal,

00:08:05.170 --> 00:08:07.250
so it's going to look like that,
and then that is going

00:08:07.250 --> 00:08:11.860
to be equal to the 0 vector.

00:08:11.860 --> 00:08:13.910
And this is going to be a square
matrix, where this has

00:08:13.910 --> 00:08:15.840
to be n, and this has to be n.

00:08:15.840 --> 00:08:17.300
How do I know that?

00:08:17.300 --> 00:08:20.570
Because I said that x1, x2, and
all of these have to be

00:08:20.570 --> 00:08:21.780
equal to 0.

00:08:21.780 --> 00:08:23.580
So they have to be equal to 0.

00:08:23.580 --> 00:08:27.380
If I just write them as a system
of equations, if I

00:08:27.380 --> 00:08:36.809
write x1 is equal to 0, x2 is
equal to 0, x3 is equal to 0,

00:08:36.809 --> 00:08:41.419
all the way to xn
is equal to 0.

00:08:41.419 --> 00:08:44.860
This system of equations, if
I wrote it as an augmented

00:08:44.860 --> 00:08:50.370
matrix, remember this is
x1 plus 0, x2 plus 0.

00:08:50.370 --> 00:08:52.860
This as in augmented matrix, and
we've done this multiple

00:08:52.860 --> 00:08:55.160
times, it would look
like this.

00:08:55.160 --> 00:08:58.780
1, you just have a bunch of 0's,
n0's, and then the 1's

00:08:58.780 --> 00:09:01.850
would just go down the diagonal,
and then you'd have

00:09:01.850 --> 00:09:04.220
n0's right there.

00:09:04.220 --> 00:09:06.220
So that's where I'm
getting it from.

00:09:06.220 --> 00:09:11.130
If we are linearly independent,
the nullspace of

00:09:11.130 --> 00:09:14.980
A is going to be just a 0
vector, and if the nullspace

00:09:14.980 --> 00:09:18.040
of A is just a 0 vector, then
the nullspace of the reduced

00:09:18.040 --> 00:09:20.430
row echelon form is
only the 0 vector.

00:09:20.430 --> 00:09:23.320
The only solution is all
of the x's equal to 0.

00:09:23.320 --> 00:09:26.420
Which means a reduced row
echelon form of A has to

00:09:26.420 --> 00:09:29.870
essentially just be 1's down
the diaganol, with 0's

00:09:29.870 --> 00:09:31.330
everywhere else.

00:09:31.330 --> 00:09:34.040
So anyway, I just want to make
this-- this is kind of a neat

00:09:34.040 --> 00:09:35.310
by-product of an

00:09:35.310 --> 00:09:36.950
interpretation of the nullspace.

00:09:36.950 --> 00:09:38.200
Let me write that.

00:09:40.490 --> 00:09:42.830
Let me summarize our results.

00:09:42.830 --> 00:09:48.660
The nullspace of A, if it just
equals 0, then that means, you

00:09:48.660 --> 00:09:53.210
can go both ways, that's true
if and only if the column

00:09:53.210 --> 00:10:03.240
vectors of A are linearly
independent.

00:10:06.830 --> 00:10:13.366
And all of that's only true--
this is true, I was going to

00:10:13.366 --> 00:10:18.240
do a triangle, it might turn
into a square-- if x1, x2, all

00:10:18.240 --> 00:10:19.780
of these have to be equal 0.

00:10:19.780 --> 00:10:21.030
This is the only solution.

00:10:23.770 --> 00:10:31.060
And then that implies that the
reduced row echelon, and I

00:10:31.060 --> 00:10:33.600
didn't do it as precisely as
I would have liked, but the

00:10:33.600 --> 00:10:37.300
reduced row echelon form of A
is essentially going to be a

00:10:37.300 --> 00:10:42.080
square n by n matrix.

00:10:42.080 --> 00:10:45.880
And, by the way, this can only
be true if we're dealing with

00:10:45.880 --> 00:10:48.560
an n by n matrix
to begin with.

00:10:48.560 --> 00:10:50.990
And maybe I'll do that a little
bit more precisely in a

00:10:50.990 --> 00:10:51.640
future video.

00:10:51.640 --> 00:10:56.210
But then the reduced row echelon
form of A is going to

00:10:56.210 --> 00:10:59.020
have to look like this, just
a bunch of 1's down the

00:10:59.020 --> 00:11:02.200
diagonal, with 0's
everywhere else.

00:11:02.200 --> 00:11:04.420
And these all imply
each other.

00:11:04.420 --> 00:11:08.990
Now, what if the nullspace of A
contains some other vectors?

00:11:08.990 --> 00:11:12.520
Well, then we would have to say
that the column vectors of

00:11:12.520 --> 00:11:15.480
A are linearly dependent.

00:11:15.480 --> 00:11:17.240
And if they're linearly
dependent, then we wouldn't

00:11:17.240 --> 00:11:20.230
have a reduced row echelon form
of A that looked like

00:11:20.230 --> 00:11:22.570
this, you would have something
that would have some free

00:11:22.570 --> 00:11:25.880
variables that allows you to
create more solutions there.

00:11:25.880 --> 00:11:29.660
But anyway, I just wanted to
give you this angle on how you

00:11:29.660 --> 00:11:33.120
can interpret the nullspace and
how it relates to linear

00:11:33.120 --> 00:11:34.370
Independence.


Null space 2: Calculating the null space of a matrix

WEBVTT
Kind: captions
Language: en

00:00:00.330 --> 00:00:02.890
In the last video, I spoke
somewhat theoretically about

00:00:02.890 --> 00:00:04.690
what a null space is
and we showed that

00:00:04.690 --> 00:00:06.700
it is a valid subspace.

00:00:06.700 --> 00:00:08.960
But in this video let's actually
calculate the null

00:00:08.960 --> 00:00:10.380
space for a matrix.

00:00:10.380 --> 00:00:14.220
In this case, we'll calculate
the null space of matrix A.

00:00:14.220 --> 00:00:17.420
So null space is literally
just the set of all the

00:00:17.420 --> 00:00:21.230
vectors that, when I multiply A
times any of those vectors,

00:00:21.230 --> 00:00:29.560
so let me say that the vector
x1, x2, x3, x4 is a member of

00:00:29.560 --> 00:00:30.850
our null space.

00:00:30.850 --> 00:00:33.410
So when I multiply this matrix
times this vector I should get

00:00:33.410 --> 00:00:34.920
the 0 vector.

00:00:34.920 --> 00:00:37.430
I should get the vector.

00:00:37.430 --> 00:00:39.450
And just to make a few
points here, this

00:00:39.450 --> 00:00:40.850
has exactly 4 columns.

00:00:40.850 --> 00:00:47.120
This is a 3 by 4 matrix, so I've
only legitimately defined

00:00:47.120 --> 00:00:51.310
multiplication of this times a
four-component vector or a

00:00:51.310 --> 00:00:53.050
member of Rn.

00:00:53.050 --> 00:00:54.970
Let me call this X.

00:00:54.970 --> 00:00:56.440
And this is our vector X.

00:00:56.440 --> 00:00:59.700
This is a member of R4.

00:00:59.700 --> 00:01:01.260
It has four components.

00:01:01.260 --> 00:01:03.040
And then when you multiply
these, we need

00:01:03.040 --> 00:01:04.480
to produce a 0 vector.

00:01:04.480 --> 00:01:07.090
The null space is the set of
all the vectors, and when I

00:01:07.090 --> 00:01:09.350
multiply it times A, I
produce the 0 vector.

00:01:09.350 --> 00:01:10.060
And what am I going to get?

00:01:10.060 --> 00:01:12.570
I'm going to have one row times
this and that's going to

00:01:12.570 --> 00:01:14.830
be the first entry, then this
row times, that's the second

00:01:14.830 --> 00:01:15.780
entry, and then the third row.

00:01:15.780 --> 00:01:21.580
So I should have three 0's So my
0 vector is going to be the

00:01:21.580 --> 00:01:24.070
0 vector in R3.

00:01:24.070 --> 00:01:27.730
So how do we figure out the set
of all of these x's that

00:01:27.730 --> 00:01:28.580
satisfy this?

00:01:28.580 --> 00:01:31.370
Let me just write our
formal notation.

00:01:31.370 --> 00:01:38.260
The null space of A is the set
of all vectors that are a

00:01:38.260 --> 00:01:41.340
member of -- we generally say
Rn, but this is a 3 by 4

00:01:41.340 --> 00:01:43.720
matrix, so these are all the
vectors that are going to be

00:01:43.720 --> 00:01:48.730
members of R4, because I'm using
this particular A, such

00:01:48.730 --> 00:01:52.910
that my matrix A times any
of these vectors is

00:01:52.910 --> 00:01:54.530
equal to the 0 vector.

00:01:54.530 --> 00:01:59.400
In this case it's going
to be 0 vector in R3.

00:01:59.400 --> 00:02:00.420
So how do we do this?

00:02:00.420 --> 00:02:02.680
Well, this is just a straight
up linear equation.

00:02:02.680 --> 00:02:04.100
We can write it that way.

00:02:04.100 --> 00:02:06.860
If we were to actually perform
the matrix multiplication, we

00:02:06.860 --> 00:02:07.830
get 1 times x1.

00:02:07.830 --> 00:02:09.010
Let me write it here.

00:02:09.010 --> 00:02:10.800
Let me do it in a
different color.

00:02:10.800 --> 00:02:19.920
1 times x1, plus 1 times x2,
plus 1 times x3, plus 1 times

00:02:19.920 --> 00:02:26.460
x4 is equal to this 0 there.

00:02:26.460 --> 00:02:29.580
So that times that is
equal to that 0.

00:02:29.580 --> 00:02:32.500
And then this times this should
be equal to that 0.

00:02:32.500 --> 00:02:41.430
So 1 times x1, so you get x1,
plus 2 times x2, plus 3 times

00:02:41.430 --> 00:02:47.710
x3, plus 4 times x4 is going
to be equal to that 0.

00:02:47.710 --> 00:02:51.510
And then finally we have that
times this vector should be

00:02:51.510 --> 00:02:52.990
equal to that 0.

00:02:52.990 --> 00:02:58.770
So the dot product of this row
vector with this column vector

00:02:58.770 --> 00:03:00.220
should be equal to that 0.

00:03:00.220 --> 00:03:02.690
So you get 4x1.

00:03:02.690 --> 00:03:21.690
4x1 plus 3x2 plus 2x1 plus 2x3
plus x4 is equal to 0.

00:03:21.690 --> 00:03:26.570
4x1 plus 3x2 plus 2x3 plus
x4 is equal to 0.

00:03:26.570 --> 00:03:29.230
You just have to find the
solution set to this and we'll

00:03:29.230 --> 00:03:33.220
essentially have figured
out our null space.

00:03:33.220 --> 00:03:35.920
Now, we've figured out the
solution set to systems of

00:03:35.920 --> 00:03:36.770
equations like this.

00:03:36.770 --> 00:03:41.090
We have three equations
with four unknowns.

00:03:41.090 --> 00:03:41.650
We can do that.

00:03:41.650 --> 00:03:44.610
We can represent this by an
augmented matrix and then put

00:03:44.610 --> 00:03:46.370
that in reduced row
echelon form.

00:03:46.370 --> 00:03:48.080
Let's do that.

00:03:48.080 --> 00:03:50.865
I can represent this problem
as the augmented matrix.

00:03:54.750 --> 00:03:57.260
1, 1, 4.

00:03:57.260 --> 00:04:00.070
1, 2, 3.

00:04:00.070 --> 00:04:03.550
1, 3, 2.

00:04:03.550 --> 00:04:07.110
and then 1, 4, 1.

00:04:07.110 --> 00:04:10.495
And then I augment that
with the 0 vector.

00:04:10.495 --> 00:04:12.720
And the immediate thing you
should notice is we took the

00:04:12.720 --> 00:04:16.610
pain of multiplying this times
this to equal that, and we

00:04:16.610 --> 00:04:19.600
wrote this as a system of
equations, but now we want to

00:04:19.600 --> 00:04:21.980
solve the system of equations,
we're going back to the

00:04:21.980 --> 00:04:23.050
augmented matrix world.

00:04:23.050 --> 00:04:25.290
What does this augmented
matrix look like?

00:04:25.290 --> 00:04:29.010
Well, this is just our
matrix A right there.

00:04:29.010 --> 00:04:31.570
That's just matrix A right
there, that's just the 0

00:04:31.570 --> 00:04:32.880
vector right there.

00:04:32.880 --> 00:04:34.980
And to solve this, and we've
done this before, we're just

00:04:34.980 --> 00:04:38.640
going to put this augmented
matrix into row echelon form.

00:04:38.640 --> 00:04:40.450
What you're going to find is
when you put it into row

00:04:40.450 --> 00:04:43.230
echelon form, this right side's
not going to change at

00:04:43.230 --> 00:04:45.680
all, because no matter what you
multiply or subtract by,

00:04:45.680 --> 00:04:47.930
you're just doing it all times
0, so you just keep

00:04:47.930 --> 00:04:48.870
ending up with 0.

00:04:48.870 --> 00:04:51.880
So as we put this into reduced
row echelon form, were

00:04:51.880 --> 00:04:55.760
actually just putting matrix A
into reduced echelon form.

00:04:55.760 --> 00:04:59.110
So let me do that, instead
of just talking about it.

00:04:59.110 --> 00:05:04.620
So let me start off by keeping
row 1 the same.

00:05:04.620 --> 00:05:10.570
Row 1 is 1, 1, 1, 1, 0.

00:05:10.570 --> 00:05:14.630
And then I want to eliminate
this 1 right here, so let me

00:05:14.630 --> 00:05:18.820
replace row 2 with row
2 minus row 1.

00:05:18.820 --> 00:05:20.980
So 1 minus 1 is 0.

00:05:20.980 --> 00:05:23.050
2 minus 1 is 1.

00:05:23.050 --> 00:05:25.170
3 minus 1 is 2.

00:05:25.170 --> 00:05:27.400
4 minus 1 is 3.

00:05:27.400 --> 00:05:28.970
0 minus 0 is 0.

00:05:28.970 --> 00:05:31.490
You can see the 0's aren't
going to change.

00:05:31.490 --> 00:05:36.740
And then let me replace this
guy with 4 times this guy,

00:05:36.740 --> 00:05:38.480
minus this guy.

00:05:38.480 --> 00:05:39.940
So I can only get rid of this.

00:05:39.940 --> 00:05:43.610
So 4 times 1 minus 4 is 0.

00:05:43.610 --> 00:05:46.950
4 times 1 minus 3 is 1.

00:05:46.950 --> 00:05:49.920
4 times 1 minus 2 is 2.

00:05:49.920 --> 00:05:52.840
4 times 1 minus 1 is 3.

00:05:52.840 --> 00:05:56.750
4 times 0 minus 0 is 0.

00:05:56.750 --> 00:05:59.490
Now I want to get rid of, if I
want to put this in reduced

00:05:59.490 --> 00:06:03.170
row echelon form, I want
to get rid of that

00:06:03.170 --> 00:06:05.660
term and that term.

00:06:05.660 --> 00:06:08.920
So let me keep my middle
row the same.

00:06:08.920 --> 00:06:13.135
My middle row is 0, 1, 2, 3.

00:06:16.380 --> 00:06:19.330
So that's 0 on the augmented
side of it, although these 0's

00:06:19.330 --> 00:06:21.470
are never going to change, it's
really just a little bit

00:06:21.470 --> 00:06:23.780
of an exercise just to
keep writing them.

00:06:23.780 --> 00:06:26.140
And my first row, let me replace
it with the first row

00:06:26.140 --> 00:06:28.360
minus the second row, so I
can get rid of this 1.

00:06:28.360 --> 00:06:30.720
So 1 minus 0 is 1.

00:06:30.720 --> 00:06:33.410
1 minus 1 is 0.

00:06:33.410 --> 00:06:36.140
1 minus 2 is minus 1.

00:06:36.140 --> 00:06:38.340
1 minus 3 is minus 2.

00:06:38.340 --> 00:06:41.390
And 0 minus 0 is 0.

00:06:41.390 --> 00:06:45.670
And let me replace this last
row with the last row minus

00:06:45.670 --> 00:06:46.610
the middle row.

00:06:46.610 --> 00:06:48.890
So 0 minus 0 is 0.

00:06:48.890 --> 00:06:51.440
1 minus 1 is 0.

00:06:51.440 --> 00:06:52.570
2 minus 2 is 0.

00:06:52.570 --> 00:06:53.770
I think you see where
this is going.

00:06:53.770 --> 00:06:55.510
3 minus 3 is 0.

00:06:55.510 --> 00:06:58.790
And obviously 0 minus 0 is 0.

00:06:58.790 --> 00:07:04.510
So this system of equations
has been reduced, just by

00:07:04.510 --> 00:07:07.560
doing reduce row echelon
form, this problem.

00:07:07.560 --> 00:07:12.030
If I just rewrite this right
here, this can be written as a

00:07:12.030 --> 00:07:21.870
system equations of x1 minus
x3 minus x4, right?

00:07:21.870 --> 00:07:25.110
The 0 x2's is equal to 0.

00:07:25.110 --> 00:07:28.830
And then this second row right
here, there's no x1, you just

00:07:28.830 --> 00:07:37.870
have an x2, plus 2x3, plus 3x2
is equal to 0, and this

00:07:37.870 --> 00:07:41.420
obviously gives me no
information whatsoever.

00:07:41.420 --> 00:07:43.510
And so I can solve this.

00:07:43.510 --> 00:07:46.160
I can solve this for x1 and
x2, and what do I get?

00:07:46.160 --> 00:07:52.960
I get x1 is equal
to x3 plus x4.

00:07:52.960 --> 00:07:54.170
Actually, I made
a mistake here.

00:07:54.170 --> 00:08:01.000
This is x1 minus x3 minus 2
times x4 is equal to 0.

00:08:01.000 --> 00:08:05.000
So if I rewrite this, I get x1
is equal to x3 plus 2x4.

00:08:07.720 --> 00:08:11.290
And then I get x2.

00:08:11.290 --> 00:08:13.280
Let me do that in green.

00:08:13.280 --> 00:08:20.875
x2 is equal to minus
2 x3 minus 3x2.

00:08:24.090 --> 00:08:28.270
So if I wanted to write the
solution set to this equation,

00:08:28.270 --> 00:08:32.640
if I wanted to write it in terms
of this, I could write

00:08:32.640 --> 00:08:40.260
x1, x2, x3, x4 is equal to
-- what's x1 equal to?

00:08:40.260 --> 00:08:54.400
It's equal to x3 times
1 plus x4 times 2.

00:08:54.400 --> 00:08:54.720
Right?

00:08:54.720 --> 00:08:57.500
I just got this right here from
this equation right here.

00:08:57.500 --> 00:09:01.600
x1 is equal to 1 times
x3, plus 2 times x4.

00:09:01.600 --> 00:09:03.320
That's just that, right there.

00:09:03.320 --> 00:09:10.145
Now, x2 is equal to x3
times minus 2, plus

00:09:10.145 --> 00:09:13.330
x4 times minus 3.

00:09:13.330 --> 00:09:14.410
What am I doing?

00:09:14.410 --> 00:09:16.050
I'm losing track of things.

00:09:16.050 --> 00:09:22.800
This x2 right here is x2 plus
2x3 plus 3x4 is equal to 0.

00:09:22.800 --> 00:09:27.620
So x2 is equal to minus
2x3, minus 3x4.

00:09:27.620 --> 00:09:28.060
Right.

00:09:28.060 --> 00:09:28.280
Like that.

00:09:28.280 --> 00:09:31.900
Sorry, my brain isn't completely
in the problem, I'm

00:09:31.900 --> 00:09:33.730
making these silly mistakes.

00:09:33.730 --> 00:09:35.270
But I think you understand
this now.

00:09:35.270 --> 00:09:36.990
So then what is x3 equal to?

00:09:36.990 --> 00:09:40.840
Well, it's just equal
to 1 times x3,

00:09:40.840 --> 00:09:42.580
plus 0 times x4, right?

00:09:42.580 --> 00:09:43.800
x3 is equal to x3.

00:09:43.800 --> 00:09:44.770
And what's x4 equal to?

00:09:44.770 --> 00:09:48.420
It's equal to 0 times
x3 plus 1 times x4.

00:09:48.420 --> 00:09:53.220
So all of the vectors in R4,
these are a member R4, which

00:09:53.220 --> 00:09:56.840
satisfy the equation, our
original equation, ax is equal

00:09:56.840 --> 00:10:01.210
to 0, can be represented as a
linear combination of these

00:10:01.210 --> 00:10:03.980
two vectors, of those
two vectors, right?

00:10:03.980 --> 00:10:07.020
These are just random scalars
that are a member of-- We can

00:10:07.020 --> 00:10:10.210
pick any real number for x3
and we could pick any real

00:10:10.210 --> 00:10:12.810
number for x4.

00:10:12.810 --> 00:10:16.140
So our solution set is just a
linear combination of those

00:10:16.140 --> 00:10:17.200
two vectors.

00:10:17.200 --> 00:10:19.060
What's another way of
saying a linear

00:10:19.060 --> 00:10:21.130
combination of two vectors?

00:10:21.130 --> 00:10:22.470
Let me write this.

00:10:22.470 --> 00:10:27.530
The null space of A, which is
just a solution set of this

00:10:27.530 --> 00:10:30.600
equation, it's just all the
x's that satisfy this

00:10:30.600 --> 00:10:34.090
equation, it equals all of the
linear combinations of this

00:10:34.090 --> 00:10:35.530
vector and that vector.

00:10:35.530 --> 00:10:38.780
What do we call all the linear
combinations of two vectors?

00:10:38.780 --> 00:10:41.330
It's the span of those
two vectors.

00:10:41.330 --> 00:10:45.000
So it equals the span of that
vector and that vector.

00:10:45.000 --> 00:10:52.290
Of the vector 1, minus 2,
1, 0, and the vector

00:10:52.290 --> 00:10:57.840
2, minus 3, 0, 1.

00:10:57.840 --> 00:11:00.490
And this is our null space.

00:11:00.490 --> 00:11:03.050
Before letting you go, let me
just point out one interesting

00:11:03.050 --> 00:11:04.640
thing right here.

00:11:04.640 --> 00:11:08.950
We represented our system of
equation like this and we put

00:11:08.950 --> 00:11:11.130
it into reduced row echelon
form, so this is

00:11:11.130 --> 00:11:12.700
A and this is 0.

00:11:12.700 --> 00:11:16.840
This right here is, let me make
sure I have some space,

00:11:16.840 --> 00:11:17.600
let me put it right here.

00:11:17.600 --> 00:11:22.970
That right there is the reduced
row echelon form of A.

00:11:22.970 --> 00:11:26.180
And so where essentially this
equation, this is a linear

00:11:26.180 --> 00:11:29.170
equation that is trying
to solve this problem.

00:11:29.170 --> 00:11:35.050
The reduced row echelon form
of A times our vector x is

00:11:35.050 --> 00:11:36.530
equal to 0.

00:11:36.530 --> 00:11:40.580
So, all the solutions to this
are also the solutions to our

00:11:40.580 --> 00:11:44.290
original problem, to our
original ax is equal to 0.

00:11:44.290 --> 00:11:46.530
So what's the solution
to this?

00:11:46.530 --> 00:11:49.320
All the x's that satisfy this,
these are the null space of

00:11:49.320 --> 00:11:52.470
the reduced row echelon
form of A.

00:11:52.470 --> 00:11:53.110
Right?

00:11:53.110 --> 00:11:57.570
So here are all of the x's, this
is the null space, this

00:11:57.570 --> 00:12:02.180
problem, if we find all of the
x's here, this is the null

00:12:02.180 --> 00:12:06.315
space of the reduced row echelon
form of our matrix A.

00:12:06.315 --> 00:12:10.030
But we're saying that this
problem is the same problem as

00:12:10.030 --> 00:12:11.790
this one, right?

00:12:11.790 --> 00:12:17.800
So we can write that the null
space of A is equal to the

00:12:17.800 --> 00:12:22.350
null space of the reduced
row echelon form of A.

00:12:22.350 --> 00:12:25.000
And that might seem a little
bit confusing, hey, why are

00:12:25.000 --> 00:12:27.430
you even writing this out, but
it's the actually very useful

00:12:27.430 --> 00:12:29.860
when you're trying to calculate
null spaces.

00:12:29.860 --> 00:12:31.240
So we didn't even have
to write a big

00:12:31.240 --> 00:12:32.180
augmented matrix here.

00:12:32.180 --> 00:12:35.750
We can say, take our matrix A,
put it in reduced row echelon

00:12:35.750 --> 00:12:37.780
form and then figure out
it's null space.

00:12:37.780 --> 00:12:40.060
We would have gone straight
to this point right here.

00:12:40.060 --> 00:12:43.170
This is the reduced row echelon
form of A, and then I

00:12:43.170 --> 00:12:46.950
could have immediately solved
these equations, right?

00:12:46.950 --> 00:12:49.040
I would have just taken the dot
product of the reduced row

00:12:49.040 --> 00:12:51.840
echelon form or, not the dot
product, the matrix vector

00:12:51.840 --> 00:12:55.350
product of the reduced row
echelon form of A with this

00:12:55.350 --> 00:12:57.800
vector, and I would've gotten
these equations, and then

00:12:57.800 --> 00:13:00.150
these equations would
immediately, I can just

00:13:00.150 --> 00:13:02.060
rewrite them in this
form, and I would

00:13:02.060 --> 00:13:03.890
have gotten our result.

00:13:03.890 --> 00:13:07.180
But anyway, hopefully you found
that reasonably useful.


Introduction to the null space of a matrix

WEBVTT
Kind: captions
Language: en

00:00:00.620 --> 00:00:03.940
Let's review our notions
of subspaces again.

00:00:03.940 --> 00:00:05.870
And then let's see if we can
define some interesting

00:00:05.870 --> 00:00:09.230
subspaces dealing with
matrices and vectors.

00:00:09.230 --> 00:00:16.040
So a subspace-- let's say that I
have some subspace-- oh, let

00:00:16.040 --> 00:00:18.210
me just call it some
subspace s.

00:00:18.210 --> 00:00:20.560
This is a subspace if the
following are true-- and this

00:00:20.560 --> 00:00:25.670
is all a review-- that the 0
vector-- I'll just do it like

00:00:25.670 --> 00:00:28.330
that-- the 0 vector,
is a member of s.

00:00:28.330 --> 00:00:30.190
So it contains the 0 vector.

00:00:30.190 --> 00:00:37.480
Then if v1 and v2 are both
members of my subspace, then

00:00:37.480 --> 00:00:43.330
v1 plus v2 is also a member
of my subspace.

00:00:43.330 --> 00:00:45.580
So that's just saying that
the subspaces are

00:00:45.580 --> 00:00:46.530
closed under addition.

00:00:46.530 --> 00:00:48.680
You can add any of their two
members and you'll get another

00:00:48.680 --> 00:00:50.280
member of the subspace.

00:00:50.280 --> 00:00:52.560
And then the last requirement,
if you remember, is that

00:00:52.560 --> 00:00:54.950
subspaces are closed under
multiplication.

00:00:54.950 --> 00:01:00.320
So that if c is real number,
and it's just a scalar.

00:01:00.320 --> 00:01:05.400
And if I multiply, and v1 is a
member of my subspace, then if

00:01:05.400 --> 00:01:10.250
I multiply that arbitrary real
number times my member of my

00:01:10.250 --> 00:01:13.380
subspace, v1, I'm going
to get another

00:01:13.380 --> 00:01:14.110
member of the subspace.

00:01:14.110 --> 00:01:16.150
So it's closed under
multiplication.

00:01:16.150 --> 00:01:18.340
These were all of what
a subspace is.

00:01:18.340 --> 00:01:20.110
This is our definition
of a subspace.

00:01:20.110 --> 00:01:21.440
If you call something
a subspace,

00:01:21.440 --> 00:01:22.940
these need to be true.

00:01:22.940 --> 00:01:25.280
Now let's see if we can do
something interesting with

00:01:25.280 --> 00:01:28.720
what we understand about matrix
vector multiplication.

00:01:28.720 --> 00:01:34.340
Let's say I have the matrix
a-- I'll make it nice and

00:01:34.340 --> 00:01:38.060
bold-- and it's an
m by n matrix.

00:01:38.060 --> 00:01:40.510
And I'm interested in the
following situation; I want to

00:01:40.510 --> 00:01:44.280
set up the homogeneous
equation.

00:01:44.280 --> 00:01:46.660
And we'll talk about why
it's homogeneous.

00:01:46.660 --> 00:01:47.760
Well, I'll tell you
in a second.

00:01:47.760 --> 00:01:49.500
So let's say we set
up the equation.

00:01:49.500 --> 00:02:00.400
My matrix a times vector x
is equal to the 0 vector.

00:02:00.400 --> 00:02:02.390
This is a homogeneous equation,

00:02:02.390 --> 00:02:03.740
because we have a 0 there.

00:02:07.480 --> 00:02:08.740
And I want to ask
the question-- I

00:02:08.740 --> 00:02:10.870
talked about subspaces.

00:02:10.870 --> 00:02:15.650
If I take all of the x's--
if I take the world, the

00:02:15.650 --> 00:02:20.360
universe, the set of all of
the x's that satisfy this

00:02:20.360 --> 00:02:24.420
equation, do I have
a valid subspace?

00:02:24.420 --> 00:02:25.470
Let's think about this.

00:02:25.470 --> 00:02:30.540
I want to take all of the x's
that are a member of Rn.

00:02:30.540 --> 00:02:35.450
Remember, if our matrix a has
n columns, then I've only

00:02:35.450 --> 00:02:39.090
defined this matrix vector
multiplication.

00:02:39.090 --> 00:02:42.620
If x is a member of r, and if
x has to have exactly n

00:02:42.620 --> 00:02:44.720
components, only then
is it defined.

00:02:44.720 --> 00:02:47.720
So let me define a set of all
the vectors that are a member

00:02:47.720 --> 00:02:54.420
of Rn where they satisfy the
equation a times my vector x

00:02:54.420 --> 00:02:57.600
is equal to the 0 vector.

00:02:57.600 --> 00:03:00.820
So my question is, is
this a subspace?

00:03:00.820 --> 00:03:04.580
Is this a valid subspace?

00:03:04.580 --> 00:03:07.730
So the first question is, does
it contain the 0 vector?

00:03:07.730 --> 00:03:10.430
Well in order for this to
contain the 0 vector, the 0

00:03:10.430 --> 00:03:13.390
vector must satisfy
this equation.

00:03:13.390 --> 00:03:19.890
So what is any m by n matrix
a times the 0 vector?

00:03:22.920 --> 00:03:27.770
Let's write out my matrix
a-- my matrix a, a[1,1]

00:03:27.770 --> 00:03:28.820
a[1,2]

00:03:28.820 --> 00:03:31.580
all the way to a[1,n]

00:03:31.580 --> 00:03:33.810
and then this, as we go down
a column, we go all

00:03:33.810 --> 00:03:35.470
the way down to a[m,1]

00:03:35.470 --> 00:03:37.110
and then as we go all the
way to the bottom

00:03:37.110 --> 00:03:39.100
right, we go to a[m,n]

00:03:39.100 --> 00:03:45.740
and I'm going to multiply that
times the 0 vector that has

00:03:45.740 --> 00:03:48.850
exactly n components.

00:03:48.850 --> 00:03:52.680
So the 0 vector with n
components is 0, 0, and you're

00:03:52.680 --> 00:03:53.940
going to have n of these.

00:03:53.940 --> 00:03:56.165
The number of components here
has to be the exact same

00:03:56.165 --> 00:03:58.560
number of the number of columns
you have. But when you

00:03:58.560 --> 00:04:02.100
take this product, this
matrix vector

00:04:02.100 --> 00:04:04.370
product, what do you get?

00:04:04.370 --> 00:04:06.790
What do we get?

00:04:06.790 --> 00:04:09.500
Well, this first term up here
is going to be a[1,1]

00:04:09.500 --> 00:04:11.000
time 0, plus a[1,2]

00:04:11.000 --> 00:04:13.820
times 0, plus each of
these terms times 0.

00:04:13.820 --> 00:04:15.780
And you add them
all up. a[1,1]

00:04:15.780 --> 00:04:16.500
times 0, plus a[1,2]

00:04:16.500 --> 00:04:16.800
plus a[1,2]

00:04:16.800 --> 00:04:18.959
times 0, all the way to a[1,n]

00:04:18.959 --> 00:04:20.470
and times 0.

00:04:20.470 --> 00:04:22.130
So you get 0.

00:04:22.130 --> 00:04:25.540
Now this term is going
to be a[2,1]

00:04:25.540 --> 00:04:27.990
times 0, plus a[2,2]

00:04:27.990 --> 00:04:29.770
times 0, plus a[2,3]

00:04:29.770 --> 00:04:31.850
times 0, all the way to a[2,n]

00:04:31.850 --> 00:04:32.560
times 0.

00:04:32.560 --> 00:04:33.880
That's, obviously,
going to be 0.

00:04:33.880 --> 00:04:36.630
And you're going to keep doing
that because all of these are,

00:04:36.630 --> 00:04:39.880
essentially,-- you can kind of
view it as the dot product

00:04:39.880 --> 00:04:43.800
of-- I haven't defined dot
products with row vectors and

00:04:43.800 --> 00:04:47.710
column vectors, but I think you
get the idea-- the sum of

00:04:47.710 --> 00:04:50.370
each of these elements,
multiplied with the

00:04:50.370 --> 00:04:52.890
corresponding component
in this vector.

00:04:52.890 --> 00:04:54.860
And, of course, you're just
always multiplying by 0 and

00:04:54.860 --> 00:04:55.340
then adding up.

00:04:55.340 --> 00:04:58.270
So you're going to get nothing
but a bunch of 0's.

00:04:58.270 --> 00:05:01.280
So the 0 vector does satisfied
the equation.

00:05:01.280 --> 00:05:04.860
A times the 0 vector is
equal to the 0 vector.

00:05:04.860 --> 00:05:06.510
And this is a very
unconventional notation.

00:05:06.510 --> 00:05:08.210
I'm just writing it like that,
because I don't feel like

00:05:08.210 --> 00:05:10.710
bolding out my 0's all the time
to make you realize that

00:05:10.710 --> 00:05:11.740
that's a vector.

00:05:11.740 --> 00:05:14.550
So we meet our first
requirement.

00:05:14.550 --> 00:05:17.360
The 0 vector is a member
of the set.

00:05:17.360 --> 00:05:21.710
So let me define my set here.

00:05:21.710 --> 00:05:22.500
Let me define it n.

00:05:22.500 --> 00:05:25.120
And I'll tell you in a second
why I'm calling it n.

00:05:25.120 --> 00:05:29.730
So we now know that
the 0 vector is a

00:05:29.730 --> 00:05:32.440
member of my set n.

00:05:32.440 --> 00:05:37.230
Now let's say I have two
vectors, v1 and v2 that are

00:05:37.230 --> 00:05:39.080
members-- let me write this.

00:05:39.080 --> 00:05:45.750
So let's say I have two factors,
v1, and v2, that are

00:05:45.750 --> 00:05:48.620
both members of our set.

00:05:48.620 --> 00:05:49.740
What does that mean?

00:05:49.740 --> 00:05:51.950
That means that they both
satisfy this equation.

00:05:51.950 --> 00:05:57.360
So that means that a-- my matrix
a-- times vector 1 is

00:05:57.360 --> 00:05:58.160
equal to 0.

00:05:58.160 --> 00:05:59.520
This is by definition.

00:05:59.520 --> 00:06:01.440
I'm saying that they're a member
of the set, which means

00:06:01.440 --> 00:06:02.910
they must satisfy this.

00:06:02.910 --> 00:06:07.280
And that also means that
a times vector 2 is

00:06:07.280 --> 00:06:10.070
equal to our 0 vector.

00:06:10.070 --> 00:06:17.720
So in order for this to be
closed under addition, a times

00:06:17.720 --> 00:06:22.470
vector 1 plus vector 2, the
sum of these two vectors

00:06:22.470 --> 00:06:23.910
should also be a member of n.

00:06:23.910 --> 00:06:25.340
But let's figure out
what this is.

00:06:25.340 --> 00:06:28.010
The sum of these two vectors
is this vector right here.

00:06:28.010 --> 00:06:29.060
This is equal to--
and I haven't

00:06:29.060 --> 00:06:30.130
proven this to you yet.

00:06:30.130 --> 00:06:31.710
I haven't made a video
where I prove this.

00:06:31.710 --> 00:06:34.110
But it's very easy to prove just
using the definition of

00:06:34.110 --> 00:06:36.510
matrix vector multiplication,
that matrix vector

00:06:36.510 --> 00:06:40.030
multiplication does display
the distributive property.

00:06:40.030 --> 00:06:42.740
And maybe I'll make a video on
that, but literally, you just

00:06:42.740 --> 00:06:44.270
have to go through the mechanics
of each of the

00:06:44.270 --> 00:06:45.340
terms.

00:06:45.340 --> 00:06:48.580
This is equal to a[v,1]

00:06:48.580 --> 00:06:50.720
plus a[v,2]

00:06:50.720 --> 00:06:53.360
And we know that this is
equal to the 0 vector.

00:06:53.360 --> 00:06:55.460
And this is equal
to the 0 vector.

00:06:55.460 --> 00:06:58.770
And if you add the 0 vector to
itself, this whole thing is

00:06:58.770 --> 00:07:01.900
going to be equal
to the 0 vector.

00:07:01.900 --> 00:07:05.940
So if v1 is a member of n, and
v2 is a member of n, which

00:07:05.940 --> 00:07:09.680
means they both satisfy this
equation, then v1 plus v2 is

00:07:09.680 --> 00:07:10.940
definitely still
a member of n.

00:07:10.940 --> 00:07:13.110
Because when I multiply
a times that, I get

00:07:13.110 --> 00:07:14.830
the 0 vector again.

00:07:14.830 --> 00:07:18.120
So let me write that
result, as well.

00:07:25.390 --> 00:07:31.140
So we now know that v1 plus
v2 is also a member of n.

00:07:31.140 --> 00:07:33.780
And the last thing we have to
show is that it's closed under

00:07:33.780 --> 00:07:35.420
multiplication.

00:07:35.420 --> 00:07:41.930
Let's say that v1 is a member
of our space that I defined

00:07:41.930 --> 00:07:44.440
here, where they satisfy
this equation.

00:07:44.440 --> 00:07:51.240
What about c times v1?

00:07:51.240 --> 00:07:53.370
Is that a member of n?

00:07:53.370 --> 00:07:54.350
Well let's think about it.

00:07:54.350 --> 00:07:59.400
What's our matrix a times
the vector-- right?

00:07:59.400 --> 00:08:00.690
I'm just multiplying this
times the scale.

00:08:00.690 --> 00:08:02.940
I'm just going to get
another vector.

00:08:02.940 --> 00:08:04.580
I don't want to write
a capital v there.

00:08:04.580 --> 00:08:06.850
Lowercase v, so it's a vector.

00:08:06.850 --> 00:08:08.300
What's this equal to?

00:08:08.300 --> 00:08:10.600
Well, once again, I haven't
prove it to you yet, but it's

00:08:10.600 --> 00:08:12.660
actually a very straightforward
thing to do,

00:08:12.660 --> 00:08:16.820
to show that when you're dealing
with scalars, if you

00:08:16.820 --> 00:08:19.140
have a scalar here, it doesn't
matter if you multiply the

00:08:19.140 --> 00:08:23.750
scalar times the vector before
multiplying it times the

00:08:23.750 --> 00:08:25.860
matrix or multiplying the matrix
times the vector, and

00:08:25.860 --> 00:08:26.790
then doing the scalar.

00:08:26.790 --> 00:08:31.210
So it's fairly straightforward
to prove that this is equal to

00:08:31.210 --> 00:08:36.360
c times our matrix a-- I'll make
that nice and bold, times

00:08:36.360 --> 00:08:37.980
our vector v.

00:08:37.980 --> 00:08:39.720
That these two things
are equivalent.

00:08:39.720 --> 00:08:42.940
Maybe I should just churn out
the video that does this, but

00:08:42.940 --> 00:08:44.210
I'll leave it to you.

00:08:44.210 --> 00:08:45.890
You, literally, just go
through the mechanics

00:08:45.890 --> 00:08:46.620
component by component.

00:08:46.620 --> 00:08:48.560
And you show this.

00:08:48.560 --> 00:08:56.130
But clearly, if there's is true,
we already know that v1

00:08:56.130 --> 00:09:00.460
is a member of our set, which
means that a times v1 is equal

00:09:00.460 --> 00:09:02.130
to the 0 vector.

00:09:02.130 --> 00:09:06.410
And so that means this will
reduce to c times the 0

00:09:06.410 --> 00:09:09.000
vector, which is still
the 0 vector.

00:09:09.000 --> 00:09:10.860
So c[v,1]

00:09:10.860 --> 00:09:13.430
is definitely a member of n.

00:09:13.430 --> 00:09:15.150
So it's closed under
multiplication.

00:09:15.150 --> 00:09:17.960
And I kind of assumed
this right here.

00:09:17.960 --> 00:09:19.660
But maybe I'll prove that
in a different video.

00:09:19.660 --> 00:09:23.110
But I want to do all this to
show that this set n is a

00:09:23.110 --> 00:09:25.380
valid subspace.

00:09:25.380 --> 00:09:26.640
This is a valid subspace.

00:09:26.640 --> 00:09:28.190
It contains a 0 vector.

00:09:28.190 --> 00:09:29.560
It's close under addition.

00:09:29.560 --> 00:09:31.380
It's close under
multiplication.

00:09:31.380 --> 00:09:33.380
And we actually have a special
name for this.

00:09:33.380 --> 00:09:46.240
We call this right here, we call
n, the null space of a.

00:09:46.240 --> 00:09:49.590
Or we could write n is equal
to-- maybe I shouldn't have

00:09:49.590 --> 00:09:50.090
written an n.

00:09:50.090 --> 00:09:52.550
Let me write orange in there.

00:09:52.550 --> 00:09:58.430
Our orange n is equal to-- the
notation is just the null

00:09:58.430 --> 00:09:59.360
space of a.

00:09:59.360 --> 00:10:01.180
Or we could write the null space
is equal to the orange

00:10:01.180 --> 00:10:04.050
notation of n, and literally,
if I just give you some

00:10:04.050 --> 00:10:08.010
arbitrary matrix a, and
I say, hey, find me n

00:10:08.010 --> 00:10:09.810
of a, what is that?

00:10:09.810 --> 00:10:16.500
Literally, your goal is to find
the set of all x's that

00:10:16.500 --> 00:10:20.020
satisfy the equation a times
x is equal to 0.

00:10:20.020 --> 00:10:22.790
And I'm going to do that
in the next video.


Matrix vector products

WEBVTT
Kind: captions
Language: en

00:00:00.000 --> 00:00:03.410
In the last couple of videos,
I already exposed you to the

00:00:03.410 --> 00:00:08.210
idea of a matrix, which is
really just an array of

00:00:08.210 --> 00:00:10.570
numbers, usually a 2-dimensional
array.

00:00:10.570 --> 00:00:11.500
Actually it's always
a 2-dimensional

00:00:11.500 --> 00:00:13.220
array for our purposes.

00:00:13.220 --> 00:00:18.670
So if I have an m by n matrix,
the m is just the number of

00:00:18.670 --> 00:00:24.960
rows, and then the n is just
the number of columns.

00:00:24.960 --> 00:00:27.565
So let me write out
the m by n matrix.

00:00:27.565 --> 00:00:32.329
So I'll just specify, let's have
the m by n matrix A, it's

00:00:32.329 --> 00:00:35.610
a capital bold A.

00:00:35.610 --> 00:00:40.050
And it is equal to, I'll be as
general as possible, first

00:00:40.050 --> 00:00:44.180
entry is in, I'll just call that
lowercase a, it's in row

00:00:44.180 --> 00:00:45.730
1 column 1.

00:00:45.730 --> 00:00:49.720
The next entry is
row 1 column 2.

00:00:49.720 --> 00:00:54.090
And you go all the way
to row 1 column

00:00:54.090 --> 00:00:56.120
n, you have n columns.

00:00:56.120 --> 00:00:59.300
And then when you go down, you
go to the next row, it will be

00:00:59.300 --> 00:01:01.640
row 2 column 1.

00:01:01.640 --> 00:01:03.170
And then you keep going
all the way down to

00:01:03.170 --> 00:01:05.680
row m column n.

00:01:05.680 --> 00:01:07.410
And then of course, what?

00:01:07.410 --> 00:01:10.140
This entry is going to be, row
2, let me write that a little

00:01:10.140 --> 00:01:13.690
smaller, row 2 column 2.

00:01:13.690 --> 00:01:16.560
And you go all the way, and
you're going to have

00:01:16.560 --> 00:01:19.000
row m column n.

00:01:19.000 --> 00:01:21.990
And so if you think about it,
you're going to have how many

00:01:21.990 --> 00:01:22.630
total entries here?

00:01:22.630 --> 00:01:25.010
You're going to have m entries
this way, n that way.

00:01:25.010 --> 00:01:27.730
So you're going to m times
n total entries.

00:01:27.730 --> 00:01:29.500
And I think you're pretty
familiar with this idea

00:01:29.500 --> 00:01:32.440
already of a matrix, you
probably saw this in your

00:01:32.440 --> 00:01:34.180
Algebra II classes.

00:01:34.180 --> 00:01:36.860
So what we want to do now in
this video is relate our

00:01:36.860 --> 00:01:40.600
notion of a matrix to everything
we already know

00:01:40.600 --> 00:01:41.870
about vectors.

00:01:41.870 --> 00:01:45.500
Or maybe introduce some
operations that allow matrix

00:01:45.500 --> 00:01:48.130
and vectors to interact
with each other.

00:01:48.130 --> 00:01:52.100
And maybe the most natural one
is multiplication, or taking

00:01:52.100 --> 00:01:53.570
the product.

00:01:53.570 --> 00:01:59.550
So what I'm going to do in this
video is define what it

00:01:59.550 --> 00:02:04.460
means when we take the product
of our matrix A, of any matrix

00:02:04.460 --> 00:02:06.880
A, I've written this as
general as possible,

00:02:06.880 --> 00:02:10.130
with some vector x.

00:02:10.130 --> 00:02:17.250
And our definition will only
work if x, the vector we're

00:02:17.250 --> 00:02:20.780
multiplying A by, has
the same number of

00:02:20.780 --> 00:02:25.920
components as A has columns.

00:02:25.920 --> 00:02:35.370
So this is only valid for an x
that looks like this: x1, x2,

00:02:35.370 --> 00:02:38.465
all the way down to x n.

00:02:38.465 --> 00:02:44.380
So let me be very clear with
this, this vector, I guess you

00:02:44.380 --> 00:02:46.000
could do it a different height
than this vector.

00:02:46.000 --> 00:02:48.550
What matters is that the same
number of A's you have in this

00:02:48.550 --> 00:02:50.880
direction, you have n A's
here, then you have n

00:02:50.880 --> 00:02:53.430
components of this vector
right here.

00:02:53.430 --> 00:02:56.080
And if you have that constraint,
if the length of

00:02:56.080 --> 00:02:59.430
your vector, or the number of
components in vector is equal

00:02:59.430 --> 00:03:02.640
to the number of columns in your
matrix, then we define

00:03:02.640 --> 00:03:09.080
this product to be equal to --
so this is my vector x -- so

00:03:09.080 --> 00:03:10.230
this is a definition.

00:03:10.230 --> 00:03:12.610
There's nothing in nature that
told us it had to be

00:03:12.610 --> 00:03:13.430
defined this way.

00:03:13.430 --> 00:03:16.770
It's just human beings, or
mathematicians, decided that

00:03:16.770 --> 00:03:20.180
this is a useful convention to
the define the multiplication,

00:03:20.180 --> 00:03:23.180
or the product, of a matrix
and a vector.

00:03:23.180 --> 00:03:29.360
So we'll define A times
our vector x.

00:03:29.360 --> 00:03:32.780
These are both bold, this is
a matrix, that's a vector.

00:03:32.780 --> 00:03:34.420
And the convention, if I didn't
draw the little vector

00:03:34.420 --> 00:03:36.620
symbol, your textbooks would
just bold out the x, so that

00:03:36.620 --> 00:03:37.780
it'll be a lowercase x.

00:03:37.780 --> 00:03:40.970
Lower case is vector, uppercase
is matrix, both of

00:03:40.970 --> 00:03:41.670
them are bolded.

00:03:41.670 --> 00:03:42.960
That tells you that you're
not just dealing

00:03:42.960 --> 00:03:44.290
with regular numbers.

00:03:44.290 --> 00:03:49.220
So we're defining this to be
equal to -- let me write it

00:03:49.220 --> 00:03:51.732
out fairly large.

00:03:51.732 --> 00:03:56.070
You're going to take each row,
and we're going to show you

00:03:56.070 --> 00:03:59.200
that there's multiple ways to
kind of visualize this, but

00:03:59.200 --> 00:04:03.350
it's going to a11 times x1,
let me write that down.

00:04:15.030 --> 00:04:23.490
So a11 times x1 plus a12 times
x2, all the way to

00:04:23.490 --> 00:04:30.050
plus a1n times xn.

00:04:32.960 --> 00:04:37.040
So the product of this matrix,
this m by n matrix and this n

00:04:37.040 --> 00:04:41.860
component vector, will be a new
vector, the first entry of

00:04:41.860 --> 00:04:44.870
which is essentially each
of these entries times a

00:04:44.870 --> 00:04:47.310
corresponding entry here,
and you add them all up.

00:04:47.310 --> 00:04:49.470
And as you can see, that's
already looking fairly similar

00:04:49.470 --> 00:04:51.400
to a dot product, and I'll
discuss that in a second.

00:04:51.400 --> 00:04:54.280
But let me finish my definition
before I start

00:04:54.280 --> 00:04:56.010
talking about what it
means, or what it

00:04:56.010 --> 00:04:56.920
might be related to.

00:04:56.920 --> 00:04:59.660
So that was that first row
right there, it'll

00:04:59.660 --> 00:05:01.550
just look like that.

00:05:01.550 --> 00:05:03.870
We just multiply that
times this thing to

00:05:03.870 --> 00:05:05.200
get that row there.

00:05:05.200 --> 00:05:08.780
Now the second row -- I want to
do it in a different color

00:05:08.780 --> 00:05:12.370
-- remember this is
a definition.

00:05:12.370 --> 00:05:13.540
Human beings came
up with this.

00:05:13.540 --> 00:05:15.720
Nothing about nature said we
had to do it this way, but

00:05:15.720 --> 00:05:17.150
it's just nice and convenient.

00:05:17.150 --> 00:05:22.770
So our second row will have a21
times x1, we'll just do

00:05:22.770 --> 00:05:24.300
the whole thing over again,
but this time we're

00:05:24.300 --> 00:05:27.560
multiplying this row times
this column vector.

00:05:27.560 --> 00:05:42.095
So a21 times x1 plus a22 times
x2 all the way until we get to

00:05:42.095 --> 00:05:49.620
-- I wanted to do that in
magenta -- a2n times xn.

00:05:49.620 --> 00:05:52.870
So we multiplied this entire row
times that entire column.

00:05:52.870 --> 00:05:55.520
This term times that term, plus
this term plus this term.

00:05:55.520 --> 00:06:00.530
All the way down to plus
this last term

00:06:00.530 --> 00:06:01.840
times that last term.

00:06:01.840 --> 00:06:05.870
And we keep doing this for every
row until we get to the

00:06:05.870 --> 00:06:12.450
m-th row, and then the
m-th row will be am1.

00:06:17.060 --> 00:06:19.260
This is the m-th row
first column.

00:06:19.260 --> 00:06:26.320
am1 times x1 plus -- it's hard
to keep switching colors --

00:06:26.320 --> 00:06:37.550
plus am2 times x2, all the way
until we get to amn times xn.

00:06:41.400 --> 00:06:43.210
So what is this vector
going to look like?

00:06:43.210 --> 00:06:48.020
It's essentially going to have
-- let's say we call this

00:06:48.020 --> 00:06:51.200
vector-- Let's say it's
equal to vector b.

00:06:51.200 --> 00:06:53.240
What does vector b look like?

00:06:53.240 --> 00:06:54.890
How many entries is
it going to have?

00:06:54.890 --> 00:06:57.060
Well it has an entry for each
row of this, right?

00:06:57.060 --> 00:07:00.090
We're taking each row and we're
essentially taking the

00:07:00.090 --> 00:07:03.710
dot product of this row vector
with this column vector.

00:07:03.710 --> 00:07:05.070
And I'll be a little bit
more formal with the

00:07:05.070 --> 00:07:06.480
notation in a second.

00:07:06.480 --> 00:07:09.660
But I think you understand that
this is a dot product.

00:07:09.660 --> 00:07:11.720
The first component times the
first component plus the

00:07:11.720 --> 00:07:14.280
second component times the
second component plus the

00:07:14.280 --> 00:07:16.180
third component times the third
component, all the way

00:07:16.180 --> 00:07:19.000
to the n-th component plus
the n-th component

00:07:19.000 --> 00:07:20.200
times the n-th component.

00:07:20.200 --> 00:07:22.555
So this is essentially the dot
product of this row vector.

00:07:25.910 --> 00:07:28.230
We've been writing all of our
vectors as columns, so we

00:07:28.230 --> 00:07:29.750
could call them column vectors,
you're just writing

00:07:29.750 --> 00:07:30.730
them as rows.

00:07:30.730 --> 00:07:32.250
And we can be a little bit
more specific with the

00:07:32.250 --> 00:07:34.960
notation in a second, but what's
this going look like?

00:07:34.960 --> 00:07:36.830
Well we're doing this
m times, so we're

00:07:36.830 --> 00:07:38.010
going to have m entries.

00:07:38.010 --> 00:07:43.660
You're going to b1 b2
all the way to bn.

00:07:43.660 --> 00:07:47.760
If you viewed these all as
matrices, you can kind of view

00:07:47.760 --> 00:07:51.880
it as -- and this will
eventually work for the matrix

00:07:51.880 --> 00:07:55.670
math we're going to learn --
this is an m by n matrix and

00:07:55.670 --> 00:07:58.730
we're multiplying it by -- how
many rows does this guy have?

00:07:58.730 --> 00:08:02.210
He has n rows.

00:08:02.210 --> 00:08:05.370
He has n components, and
he has 1 column.

00:08:05.370 --> 00:08:10.560
So m by n times an n by 1, you
essentially can ignore these

00:08:10.560 --> 00:08:13.610
middle two terms, and they'll
result with -- how many rows

00:08:13.610 --> 00:08:14.390
does this guy have?

00:08:14.390 --> 00:08:17.880
He has m rows, and 1 column.

00:08:17.880 --> 00:08:20.320
These middle two terms have to
be equal to each other just

00:08:20.320 --> 00:08:23.170
for the multiplication to be
defined, and then you're left

00:08:23.170 --> 00:08:25.960
with an m by 1 matrix.

00:08:25.960 --> 00:08:28.990
So this was all abstract, let
me actually apply it to some

00:08:28.990 --> 00:08:29.610
actual numbers.

00:08:29.610 --> 00:08:31.665
But it's important to actually
set the definition.

00:08:34.700 --> 00:08:36.559
Now that we have the definition
we can apply it to

00:08:36.559 --> 00:08:40.130
some actual matrices
and vectors.

00:08:40.130 --> 00:08:43.260
So let's say we have
the matrix.

00:08:43.260 --> 00:08:51.350
Let's say I want to multiply the
matrix minus 3, 0, 3, 2.

00:08:51.350 --> 00:08:52.740
Now I'll do this
one in yellow.

00:08:52.740 --> 00:08:57.070
1, 7, minus 1, 9.

00:08:57.070 --> 00:09:02.880
And I want to multiply
that by the vector.

00:09:02.880 --> 00:09:05.700
Now how many components,
or rows, does this

00:09:05.700 --> 00:09:07.400
vector have to have?

00:09:07.400 --> 00:09:11.620
Well my matrix times vector
product, or multiplication, is

00:09:11.620 --> 00:09:16.650
only defined if my vector has
as many components as this

00:09:16.650 --> 00:09:17.910
matrix has columns.

00:09:17.910 --> 00:09:19.350
So we have 1, 2, 3, 4 columns.

00:09:19.350 --> 00:09:21.990
So this guy's going to have 4
components for us even to be

00:09:21.990 --> 00:09:23.520
able to multiply
them, otherwise

00:09:23.520 --> 00:09:24.460
it wouldn't be defined.

00:09:24.460 --> 00:09:25.640
So let me put 4 entries here.

00:09:25.640 --> 00:09:32.750
Let's say it's 2, minus 3,
4, and then minus 1.

00:09:32.750 --> 00:09:34.580
So what is this going
to be equal to?

00:09:40.370 --> 00:09:42.730
The first term of this is going
to be the dot product of

00:09:42.730 --> 00:09:46.100
this first row with
this vector.

00:09:46.100 --> 00:09:48.360
And then the second entry is
going to be the dot product of

00:09:48.360 --> 00:09:51.980
this row vector with
this column.

00:09:51.980 --> 00:09:52.920
So let's do it.

00:09:52.920 --> 00:09:58.170
So it's going to be minus 3
times 2, I'm not going to

00:09:58.170 --> 00:10:05.990
color code it, minus 3 times 2
plus 0 times minus 3 plus 3

00:10:05.990 --> 00:10:10.600
times 4 plus 2 times minus 1.

00:10:13.540 --> 00:10:17.590
And now my second row, or I
guess my second component in

00:10:17.590 --> 00:10:23.220
this vector, is going to be 1
times 2 plus 7 times negative

00:10:23.220 --> 00:10:33.050
3 plus minus 1 times 4
plus 9 times minus 1.

00:10:35.720 --> 00:10:37.300
And so what does this
simplify to?

00:10:37.300 --> 00:10:48.460
This is equal to minus 3 times
2 is minus 6 plus 0 plus 12.

00:10:48.460 --> 00:10:49.670
This is 12.

00:10:49.670 --> 00:10:51.700
Minus 2.

00:10:51.700 --> 00:11:06.150
And then this is simplified to
2 minus 21 minus 4 minus 9.

00:11:06.150 --> 00:11:10.450
So this is equal to this top
term, let's see, I have a

00:11:10.450 --> 00:11:13.940
minus 6 plus 12 is
6 minus 2 is 4.

00:11:13.940 --> 00:11:21.020
And then I have 2 minus
21 is minus 19.

00:11:21.020 --> 00:11:23.000
I want to make sure I get
the math right here.

00:11:23.000 --> 00:11:28.350
Minus 21 minus 9 is minus 30 and
I have a minus 34 and then

00:11:28.350 --> 00:11:32.320
I have a plus 2, so minus 32.

00:11:32.320 --> 00:11:36.550
So that's my product
right there.

00:11:36.550 --> 00:11:39.010
And let me be very
clear right here.

00:11:39.010 --> 00:11:41.400
Everything we've been used to
right now, we've been writing

00:11:41.400 --> 00:11:43.610
our vectors as column vectors.

00:11:43.610 --> 00:11:51.300
But you can view each of these
right here as a row vector.

00:11:51.300 --> 00:11:52.740
But let me be even better.

00:11:52.740 --> 00:11:59.540
Let's say that vector, let
me call vector a, a1.

00:11:59.540 --> 00:12:09.500
So let me define vector a1 is
equal to minus 3, 0, 3, 2.

00:12:09.500 --> 00:12:19.570
And let me define vector a2 to
be equal to 1, 7, minus 1, 9.

00:12:19.570 --> 00:12:21.940
So all I did is I wrote these
guys, but I wrote them in our

00:12:21.940 --> 00:12:23.100
standard vector form.

00:12:23.100 --> 00:12:25.020
I wrote them as column
vectors.

00:12:25.020 --> 00:12:29.550
So what we can define to turn
these guys into row vectors is

00:12:29.550 --> 00:12:30.950
the transpose function.

00:12:30.950 --> 00:12:34.300
In transpose, you just turn the
rows into columns and the

00:12:34.300 --> 00:12:35.170
columns into rows.

00:12:35.170 --> 00:12:41.980
So if this is a1, then a1
transpose will just be the row

00:12:41.980 --> 00:12:42.920
version of this.

00:12:42.920 --> 00:12:48.180
So it's minus 3, 0, 3, 2.

00:12:48.180 --> 00:12:58.830
And then a2 transpose would be
equal to 1, 7, minus 1, and 9.

00:12:58.830 --> 00:13:01.680
And then this multiplication
right here, we can rewrite it

00:13:01.680 --> 00:13:10.650
as -- we have vector a1
transpose for the first row.

00:13:13.275 --> 00:13:16.490
These are vectors now,
row vectors.

00:13:16.490 --> 00:13:18.140
And then this is a2 transpose.

00:13:21.770 --> 00:13:24.210
The transpose should be
the super script.

00:13:24.210 --> 00:13:26.880
This vector can be written
exactly like this because this

00:13:26.880 --> 00:13:28.280
is the first row, this
is the second row.

00:13:30.840 --> 00:13:41.550
Times the vector, let me just
call this vector x, that right

00:13:41.550 --> 00:13:43.016
there is vector x.

00:13:46.770 --> 00:13:50.210
We can now rewrite the
definition as this would be

00:13:50.210 --> 00:13:51.770
equal to what?

00:13:51.770 --> 00:13:58.940
This first row right here
that we wrote out,

00:13:58.940 --> 00:14:04.550
this was a1 dot x.

00:14:04.550 --> 00:14:06.850
You know all about
the dot products.

00:14:06.850 --> 00:14:14.960
The first row was a1 dot x.

00:14:14.960 --> 00:14:20.860
It's minus 3 times 2 plus 0
times minus 3 plus 3 times 4.

00:14:20.860 --> 00:14:22.760
It's a1 dot x.

00:14:22.760 --> 00:14:25.680
And this is useful because
when I defined the dot

00:14:25.680 --> 00:14:28.540
product, I only defined it with
column vectors like this.

00:14:28.540 --> 00:14:30.720
And I'm dotting 2
column vectors.

00:14:30.720 --> 00:14:33.030
I haven't formally defined
a row vector

00:14:33.030 --> 00:14:34.430
times a column vector.

00:14:34.430 --> 00:14:38.440
So now I can say if this is just
a standard column factor,

00:14:38.440 --> 00:14:41.700
like we've been working with, I
can write my matrix as each

00:14:41.700 --> 00:14:43.770
row is the transpose
of a column vector,

00:14:43.770 --> 00:14:45.400
or it's a row vector.

00:14:45.400 --> 00:14:48.440
Then I can write this product
as just the dot products of

00:14:48.440 --> 00:14:51.160
each of these transpose, or
I guess you could say the

00:14:51.160 --> 00:14:54.030
inverse transpose, with this
vector right here.

00:14:54.030 --> 00:15:02.060
And then obviously the second
row is going to be a2 dot x.

00:15:04.610 --> 00:15:10.860
The second row is a2 dot x, is 1
times 2 plus 7 times minus 3

00:15:10.860 --> 00:15:15.370
minus 1 times 4 plus
9 times minus 1.

00:15:15.370 --> 00:15:18.950
So just like that.

00:15:18.950 --> 00:15:21.430
So this is one way to view it.

00:15:21.430 --> 00:15:24.870
Matrix times the vector is just
like the transpose of its

00:15:24.870 --> 00:15:28.090
rows dotted with the vector
you're ds it by.

00:15:28.090 --> 00:15:31.270
This is one way to perceive
matrix multiplication.

00:15:31.270 --> 00:15:37.910
Now the other way to perceive
it -- let me do it with a

00:15:37.910 --> 00:15:41.220
different example.

00:15:41.220 --> 00:15:43.900
Those numbers are getting
a little bit tiresome.

00:15:43.900 --> 00:15:55.860
Let's say I have the matrix A,
nice and bold, is equal to 3,

00:15:55.860 --> 00:16:06.560
1, 0, 3, 2, 4, 7, 0, minus
1, 2, 3, and 4.

00:16:06.560 --> 00:16:10.000
And I need to multiply this
times a 4 component vector.

00:16:10.000 --> 00:16:25.920
So let me call vector x is equal
to x1, x2, x3, and x4.

00:16:25.920 --> 00:16:29.510
Now instead of viewing these as
row vectors, we could view

00:16:29.510 --> 00:16:32.540
A as a set of column vectors.

00:16:32.540 --> 00:16:37.650
We could call this thing
right here vector 1.

00:16:37.650 --> 00:16:41.710
We call this thing right
here vector 2.

00:16:41.710 --> 00:16:45.400
We call this thing right
here vector 3.

00:16:45.400 --> 00:16:49.000
And we call this thing
right here vector 4.

00:16:49.000 --> 00:16:55.440
Then we could rewrite our matrix
A as being equal to

00:16:55.440 --> 00:16:58.380
just a bunch of column
vectors.

00:16:58.380 --> 00:17:03.640
So we could rewrite it vector
1, vector 2, vector

00:17:03.640 --> 00:17:06.709
3, and vector 4.

00:17:09.390 --> 00:17:12.380
So how can the matrix
multiplication be interpreted

00:17:12.380 --> 00:17:13.920
in this context?

00:17:13.920 --> 00:17:15.170
Well what did we do?

00:17:17.930 --> 00:17:22.079
When we multiply these guys,
all of the elements in here

00:17:22.079 --> 00:17:25.369
always get multiplied by x1.

00:17:25.369 --> 00:17:27.430
Let me start some of the
multiplication here, just from

00:17:27.430 --> 00:17:28.700
our definition.

00:17:28.700 --> 00:17:34.010
So if I multiply A times x,
I'll start it off, maybe I

00:17:34.010 --> 00:17:34.980
won't do the whole thing.

00:17:34.980 --> 00:17:35.990
I just want you to
see the pattern.

00:17:35.990 --> 00:17:45.770
It's 3 times x1 plus 1 times
x2 plus 0 times x3

00:17:45.770 --> 00:17:47.520
plus 3 times x4.

00:17:47.520 --> 00:17:48.860
That's the first entry.

00:17:48.860 --> 00:17:55.980
And then you have 2 times x1
plus 4 times x2 all the way.

00:17:55.980 --> 00:18:00.140
And then you finally have
minus 1 times x1

00:18:00.140 --> 00:18:02.210
plus 2 times x2.

00:18:02.210 --> 00:18:03.440
You get the idea.

00:18:03.440 --> 00:18:05.700
But what's happening here?

00:18:05.700 --> 00:18:08.890
This first vector is always
being multiplied

00:18:08.890 --> 00:18:10.460
by the scalar x1.

00:18:10.460 --> 00:18:15.880
In fact you can view this part
of the entries right here.

00:18:15.880 --> 00:18:20.880
We're just multiplying this guy
times the scalar of x1 in

00:18:20.880 --> 00:18:21.780
every case.

00:18:21.780 --> 00:18:23.440
You have 3, 2, minus
1, 3, 2, minus 1.

00:18:23.440 --> 00:18:24.980
We're multiplying by
the scalar of x1.

00:18:24.980 --> 00:18:30.480
And then we're adding that to
this guy times the scalar x2

00:18:30.480 --> 00:18:35.400
and then we're adding that to
this guy times the scalar x3.

00:18:35.400 --> 00:18:52.640
So we can rewrite A times x as
being equal to the scalar x1

00:18:52.640 --> 00:19:00.510
times the vector v1 plus
the scalar x2.

00:19:00.510 --> 00:19:04.700
This is the scalar x1 times the
vector v1 plus the scalar

00:19:04.700 --> 00:19:08.460
x2 times the vector v2.

00:19:08.460 --> 00:19:09.710
I want to do that in yellow.

00:19:14.100 --> 00:19:23.570
Plus x3 times the vector
v3 plus the scalar x4

00:19:23.570 --> 00:19:26.280
times the vector v4.

00:19:26.280 --> 00:19:28.290
And obviously if we had n terms
here, we'd have to have

00:19:28.290 --> 00:19:30.930
n vectors here, and we
could just make this

00:19:30.930 --> 00:19:32.090
more general to n.

00:19:32.090 --> 00:19:36.520
But what's interesting here is
now the product Ax can be

00:19:36.520 --> 00:19:40.250
interpreted as a linear
combination.

00:19:40.250 --> 00:19:43.910
These are just arbitrary numbers
depending on what our

00:19:43.910 --> 00:19:45.070
vector x is.

00:19:45.070 --> 00:19:47.310
So depending on our vector
x, we're taking a linear

00:19:47.310 --> 00:19:50.950
combination of the column
vectors of A.

00:19:50.950 --> 00:20:01.070
So this is a linear
combination of

00:20:01.070 --> 00:20:06.410
column vectors of A.

00:20:06.410 --> 00:20:07.680
So this is really interesting.

00:20:07.680 --> 00:20:10.510
I'm sure you've been exposed
to matrix multiplication in

00:20:10.510 --> 00:20:15.050
the past. But I really want you
to absorb these two ways

00:20:15.050 --> 00:20:17.100
of interpreting it, because
they'll be important when we

00:20:17.100 --> 00:20:19.490
talk about column spaces
and things

00:20:19.490 --> 00:20:20.740
like that in the future.

00:20:22.830 --> 00:20:24.680
Actually there's other ways
you can actually interpret

00:20:24.680 --> 00:20:26.800
that as a transformation
of this vector x.

00:20:26.800 --> 00:20:28.910
But I won't cover that in this
video just for brevity.

00:20:28.910 --> 00:20:33.310
But you can interpret it as a
weighted combination, or a

00:20:33.310 --> 00:20:36.710
linear combination of the column
vectors of A, where the

00:20:36.710 --> 00:20:39.360
matrix X dictates what
the weights on each

00:20:39.360 --> 00:20:40.700
of the columns are.

00:20:40.700 --> 00:20:45.360
Or you can interpret it as,
essentially, the dot product

00:20:45.360 --> 00:20:47.840
of the row vectors, or you could
define the row vectors

00:20:47.840 --> 00:20:49.670
as a transpose of
column vectors.

00:20:49.670 --> 00:20:52.250
The dot product of those column
vectors, each of the

00:20:52.250 --> 00:20:55.240
corresponding column vectors,
with your matrix X.

00:20:55.240 --> 00:20:57.750
So these are both completely
valid interpretations, and

00:20:57.750 --> 00:21:01.090
hopefully this video at least
gives you a working knowledge

00:21:01.090 --> 00:21:02.720
of what matrix multiplication
is.

00:21:02.720 --> 00:21:05.480
And even better, gives you a
little bit deeper sense of all

00:21:05.480 --> 00:21:08.390
of the different ways that
it can be interpreted.