Visualizing a projection onto a plane

WEBVTT
Kind: captions
Language: en

00:00:00.490 --> 00:00:03.320
I'm going to do one more video
where we compare old and new

00:00:03.320 --> 00:00:05.320
definitions of a projection.

00:00:05.320 --> 00:00:10.560
Our old definition of a
projection onto some line, l,

00:00:10.560 --> 00:00:18.380
of the vector, x, is the vector
in l, or that's a

00:00:18.380 --> 00:00:26.500
member of l, such that x minus
that vector, minus the

00:00:26.500 --> 00:00:38.540
projection onto l of x,
is orthogonal to l.

00:00:38.540 --> 00:00:41.990
So the visualization is, if
you have your line l like

00:00:41.990 --> 00:00:46.360
this, that is your line
l right there.

00:00:46.360 --> 00:00:49.950
And then you have some other
vector x that we're take the

00:00:49.950 --> 00:00:52.010
projection of it on to l.

00:00:52.010 --> 00:00:53.450
So that's x.

00:00:53.450 --> 00:00:56.770
The projection of x onto l,
this thing right here, is

00:00:56.770 --> 00:01:00.490
going to be some vector in l.

00:01:00.490 --> 00:01:03.060
Such that when I take the
difference between x and that

00:01:03.060 --> 00:01:05.880
vector, it's going to
be orthogonal to l.

00:01:05.880 --> 00:01:07.940
So it's going to be
some vector in l.

00:01:07.940 --> 00:01:10.190
This was our old definition when
we took the projection

00:01:10.190 --> 00:01:10.990
onto a line.

00:01:10.990 --> 00:01:12.460
Some vector in l.

00:01:12.460 --> 00:01:13.460
Maybe it's there.

00:01:13.460 --> 00:01:16.090
And if I take the difference
between that and that, this

00:01:16.090 --> 00:01:18.090
difference vector's going
to be orthogonal to

00:01:18.090 --> 00:01:19.340
everything in l.

00:01:23.080 --> 00:01:26.110
Just like that.

00:01:26.110 --> 00:01:28.700
So, this right here would be
it's difference vector.

00:01:28.700 --> 00:01:33.650
That would be x minus the
projection of x onto l.

00:01:33.650 --> 00:01:36.200
And then, of course, this
vector right here.

00:01:36.200 --> 00:01:39.950
This is the one we
were defining it.

00:01:39.950 --> 00:01:44.350
That was the projection
onto l of x.

00:01:44.350 --> 00:01:46.270
Now, what's a different
way that we could

00:01:46.270 --> 00:01:48.640
have written this?

00:01:48.640 --> 00:01:51.230
We could have written this
exact same definition.

00:01:51.230 --> 00:01:56.200
We could have said it is the
vector in l such that-- so we

00:01:56.200 --> 00:01:58.460
could say, let me write
it here in purple.

00:01:58.460 --> 00:02:14.010
Is the vector v in l such that
v-- let me write it this way--

00:02:14.010 --> 00:02:18.215
such that x minus v, right?

00:02:18.215 --> 00:02:24.340
x minus the projection of l is
orthogonal is equal to w,

00:02:24.340 --> 00:02:36.660
which is orthogonal to
everything in l.

00:02:36.660 --> 00:02:38.730
Being orthogonal to l literally
means being

00:02:38.730 --> 00:02:41.050
orthogonal to every
vector in l.

00:02:41.050 --> 00:02:43.180
So I just rewrote it a little
bit different, instead of just

00:02:43.180 --> 00:02:45.800
leaving it as a projection
of x onto l.

00:02:45.800 --> 00:02:51.780
I said hey, that's some vector,
v, in l such that x

00:02:51.780 --> 00:02:58.310
minus v is equal to some other
vector, w, which is orthogonal

00:02:58.310 --> 00:02:59.660
to everything in l.

00:02:59.660 --> 00:03:05.090
Or we can rewrite that statement
right there as x is

00:03:05.090 --> 00:03:08.250
equal to v plus w.

00:03:08.250 --> 00:03:12.670
So we can just say that the
projection of x onto l is the

00:03:12.670 --> 00:03:19.470
unique vector v in l, such that
x is equal to v plus w,

00:03:19.470 --> 00:03:23.590
where w is a unique vector-- I
mean it is going to be unique

00:03:23.590 --> 00:03:27.460
vector-- in the orthogonal
complement of l.

00:03:27.460 --> 00:03:28.080
Right?

00:03:28.080 --> 00:03:31.070
This is got to be orthogonal
to everything in l.

00:03:31.070 --> 00:03:33.410
So that's going to be a member
of the orthogonal

00:03:33.410 --> 00:03:38.070
complement of l.

00:03:38.070 --> 00:03:40.910
So this definition is actually
completely consistent with our

00:03:40.910 --> 00:03:42.950
new subspace definition.

00:03:42.950 --> 00:03:45.820
And we could just extend
it to arbitrary

00:03:45.820 --> 00:03:47.410
subspaces, not just lines.

00:03:47.410 --> 00:03:51.390
Let me help you visualize
that.

00:03:51.390 --> 00:03:55.630
So let's say we're dealing
with R3 right here.

00:03:59.590 --> 00:04:02.890
And I've got some
subspace in R3.

00:04:02.890 --> 00:04:04.950
And let's say that subspace
happens to be a plane.

00:04:04.950 --> 00:04:08.140
I'm going to make it a plane
just so that it becomes clear

00:04:08.140 --> 00:04:10.290
that we don't have to take
projections just onto lines.

00:04:10.290 --> 00:04:13.920
So this is my subspace
v right there.

00:04:13.920 --> 00:04:16.140
Let me draw its orthogonal
compliment.

00:04:16.140 --> 00:04:18.500
Let's say its orthogonal
complement looks

00:04:18.500 --> 00:04:20.610
something like that.

00:04:20.610 --> 00:04:22.100
Let's say it's a line.

00:04:22.100 --> 00:04:24.300
And then it goes-- it intersects
right there.

00:04:24.300 --> 00:04:25.520
Then it goes back.

00:04:25.520 --> 00:04:28.360
And, of course, it would have to
intersect at the 0 vector.

00:04:28.360 --> 00:04:33.170
That's the only place where a
subspace and its orthogonal

00:04:33.170 --> 00:04:34.160
complement overlap.

00:04:34.160 --> 00:04:36.710
And then it goes behind
and you see it again.

00:04:36.710 --> 00:04:38.860
Obviously you wouldn't be able
to you again because this

00:04:38.860 --> 00:04:40.540
plane would extend in
every direction.

00:04:40.540 --> 00:04:41.770
But you get the idea.

00:04:41.770 --> 00:04:45.330
So this right here
is the orthogonal

00:04:45.330 --> 00:04:48.550
complement of v, that line.

00:04:48.550 --> 00:04:53.510
Now, let's have some other
arbitrary vector in R3 here.

00:04:53.510 --> 00:04:56.850
So let's say I have some vector
that looks like that.

00:04:56.850 --> 00:04:59.180
Let's say that that is x.

00:04:59.180 --> 00:05:08.670
Now our new definition for the
projection of x onto v is

00:05:08.670 --> 00:05:16.540
equal to the unique vector v.

00:05:16.540 --> 00:05:17.390
This is a vector v.

00:05:17.390 --> 00:05:18.400
That's a subspace v.

00:05:18.400 --> 00:05:27.240
The unique vector v, that is a
member of v, such that x is

00:05:27.240 --> 00:05:43.120
equal to v plus w, where w
is a unique member of the

00:05:43.120 --> 00:05:45.610
orthogonal complement of v.

00:05:45.610 --> 00:05:47.120
This is our new definition.

00:05:47.120 --> 00:05:51.250
So, if we say x is equal to
some member of v and some

00:05:51.250 --> 00:05:53.490
member of its orthogonal
complement-- we can visually

00:05:53.490 --> 00:05:54.710
understand that here.

00:05:54.710 --> 00:05:59.280
We could say, OK it's going to
be equal to, on v, it'll be

00:05:59.280 --> 00:06:01.470
equal to that vector
to right there.

00:06:01.470 --> 00:06:04.410
And then on v's orthogonal
complement,

00:06:04.410 --> 00:06:05.670
you add that to it.

00:06:05.670 --> 00:06:07.470
So, if you were to shift it
over, you would get that

00:06:07.470 --> 00:06:09.010
vector, just like that.

00:06:09.010 --> 00:06:11.310
This right here is v.

00:06:11.310 --> 00:06:12.750
That right there is v.

00:06:12.750 --> 00:06:15.950
And then this is vector that
goes up like this, out of the

00:06:15.950 --> 00:06:18.340
plane, orthogonal to
the plane, is w.

00:06:18.340 --> 00:06:21.940
You could see if you take v plus
w, you're going to get x.

00:06:21.940 --> 00:06:29.400
And you could see that v is
the projection onto the

00:06:29.400 --> 00:06:33.610
subspace capital v-- so this
is a vector, v-- is the

00:06:33.610 --> 00:06:38.600
projection onto the subspace
capital V of the vector x.

00:06:38.600 --> 00:06:40.710
So the analogy to a shadow
still holds.

00:06:40.710 --> 00:06:43.640
If you imagine kind of a light
source coming straight down

00:06:43.640 --> 00:06:47.060
onto our subspace, kind of
orthogonal to our subspace,

00:06:47.060 --> 00:06:50.570
the projection onto our subspace
is kind of the shadow

00:06:50.570 --> 00:06:51.320
of our vector x.

00:06:51.320 --> 00:06:53.880
Hopefully that help you
visualize it a little better.

00:06:53.880 --> 00:06:57.430
But what we're doing here is
we're going to generalize it.

00:06:57.430 --> 00:06:58.960
Earlier in this video
I showed you a line.

00:06:58.960 --> 00:07:00.310
This is a plane.

00:07:00.310 --> 00:07:02.000
But we can generalize
it to any subspace.

00:07:02.000 --> 00:07:02.920
This is in R3.

00:07:02.920 --> 00:07:06.230
We can generalize it
to Rn, to R100.

00:07:06.230 --> 00:07:08.630
And that's really the power
of what we're doing here.

00:07:08.630 --> 00:07:11.360
It's easy to visualize it here,
but it's not so easy to

00:07:11.360 --> 00:07:13.530
visualize it once you get
to higher dimensions.

00:07:13.530 --> 00:07:14.900
And actually, one other thing.

00:07:14.900 --> 00:07:17.370
Let me show that this new
definition is pretty much

00:07:17.370 --> 00:07:20.130
almost identical to exactly
what we did with lines.

00:07:20.130 --> 00:07:24.950
This is identical to saying that
the projection onto the

00:07:24.950 --> 00:07:45.820
subspace x is equal to some
unique vector in V such that x

00:07:45.820 --> 00:07:58.290
minus the projection onto v of
x is orthogonal to every

00:07:58.290 --> 00:08:01.900
member of V.

00:08:01.900 --> 00:08:05.230
Because this statement, right
here, is saying any vector

00:08:05.230 --> 00:08:08.250
that's orthogonal to any member
of v says that it's a

00:08:08.250 --> 00:08:10.610
member of the orthogonal
complement of v.

00:08:10.610 --> 00:08:13.830
So that statement could be
written as x minus the

00:08:13.830 --> 00:08:20.450
projection onto v of x is a
member of v's orthogonal

00:08:20.450 --> 00:08:21.030
complement.

00:08:21.030 --> 00:08:23.310
Or we could call some w.

00:08:23.310 --> 00:08:27.210
So if you call this your v,
and if you call this whole

00:08:27.210 --> 00:08:31.640
thing your w, you get this exact
definition right there.

00:08:31.640 --> 00:08:35.820
You would have w is equal
to x minus v.

00:08:35.820 --> 00:08:39.840
And then if you add v to both
sides, you get w plus v is

00:08:39.840 --> 00:08:40.950
equal to x.

00:08:40.950 --> 00:08:44.250
We defined v to be, the
orthogonal-- the

00:08:44.250 --> 00:08:46.250
projection of x onto v.

00:08:46.250 --> 00:08:51.050
w is a member of our orthogonal
complement.

00:08:51.050 --> 00:08:53.990
And I don't want you
to get confused.

00:08:53.990 --> 00:08:57.560
The vector v is the orthogonal
projection of our vector x

00:08:57.560 --> 00:09:00.080
onto the subspace capital V.

00:09:00.080 --> 00:09:02.170
I probably should use different
letters instead of

00:09:02.170 --> 00:09:03.740
using a lowercase and
a uppercase v.

00:09:03.740 --> 00:09:05.540
It makes the language
a little difficult.

00:09:05.540 --> 00:09:07.890
But I just wanted to give you
another video to give you a

00:09:07.890 --> 00:09:10.040
visualization of projections
onto

00:09:10.040 --> 00:09:11.650
subspaces other than lines.

00:09:11.650 --> 00:09:14.570
And to show you that our old
definition, with just a

00:09:14.570 --> 00:09:17.070
projection onto a line which was
a linear transformation,

00:09:17.070 --> 00:09:19.400
is essentially equivalent
to this new definition.

00:09:19.400 --> 00:09:23.400
On the next video, I'll show
you that this, for any

00:09:23.400 --> 00:09:26.740
subspace is, indeed, a linear
transformation.


Projections onto subspaces

WEBVTT
Kind: captions
Language: en

00:00:00.580 --> 00:00:04.930
Many videos ago we introduced
the idea of a projection.

00:00:04.930 --> 00:00:08.210
And in that case we dealt more
particularly with projections

00:00:08.210 --> 00:00:10.430
onto lines that went
through the origin.

00:00:10.430 --> 00:00:14.950
So if we had some line-- let's
say L-- and let's say L is

00:00:14.950 --> 00:00:20.090
equal to the span of
some vector v.

00:00:20.090 --> 00:00:24.760
Or you could say, alternately,
that L is equal to the set of

00:00:24.760 --> 00:00:30.290
all multiples of v, such that
the scalar factors are just

00:00:30.290 --> 00:00:31.760
any real numbers.

00:00:31.760 --> 00:00:34.090
These are both representations
of lines that

00:00:34.090 --> 00:00:35.440
go through the origin.

00:00:35.440 --> 00:00:39.840
We defined a projection of any
vector onto that line.

00:00:39.840 --> 00:00:42.820
Let me just draw it real
fast. So let me see,

00:00:42.820 --> 00:00:44.160
we draw some axes.

00:00:44.160 --> 00:00:46.710
So that is my-- I want to draw
it a little bit straighter

00:00:46.710 --> 00:00:50.790
than that-- that is my vertical
axis and that is my

00:00:50.790 --> 00:00:53.210
horizontal axis.

00:00:53.210 --> 00:00:55.920
Just like that and let's say
I have some line that goes

00:00:55.920 --> 00:00:57.520
through the origin.

00:00:57.520 --> 00:00:59.950
Let's say-- that doesn't go
through the origin-- let's say

00:00:59.950 --> 00:01:02.210
that that line right there
goes through the origin.

00:01:02.210 --> 00:01:04.180
So that is L.

00:01:04.180 --> 00:01:09.560
We knew visually that a
projection of some vector x

00:01:09.560 --> 00:01:13.680
onto L-- so let's say that
that is a vector x.

00:01:13.680 --> 00:01:17.570
Visually, if you were to draw--
if you have some light

00:01:17.570 --> 00:01:21.790
coming straight down it would
be the shadow of x onto L.

00:01:21.790 --> 00:01:30.220
So this right here, that right
there, was the projection onto

00:01:30.220 --> 00:01:33.980
the line L of the vector x.

00:01:33.980 --> 00:01:35.510
And we defined it
more formally.

00:01:35.510 --> 00:01:37.670
We kind of took a
perpendicular.

00:01:37.670 --> 00:01:42.850
We said that x minus the
projection of x onto L is

00:01:42.850 --> 00:01:45.420
perpendicular to the line
L, or perpendicular to

00:01:45.420 --> 00:01:48.210
everything-- orthogonal to
everything-- on the line L.

00:01:48.210 --> 00:01:49.900
But this is how at least
I visualize this.

00:01:49.900 --> 00:01:53.870
It's kind of the shadow as you
go down onto the line L.

00:01:53.870 --> 00:01:56.890
And this was a special case,
in general, of projections.

00:01:56.890 --> 00:02:00.580
You might notice that L is going
to be a valid subspace.

00:02:00.580 --> 00:02:01.710
You could prove it
to yourself.

00:02:01.710 --> 00:02:03.050
It contains the zero vector.

00:02:03.050 --> 00:02:04.570
It goes through the origin.

00:02:04.570 --> 00:02:07.680
It's closed under addition--
any member of it plus any

00:02:07.680 --> 00:02:09.740
other member of it is going to
be another member of it.

00:02:09.740 --> 00:02:13.480
It's closed under scalar
multiplication-- you can take

00:02:13.480 --> 00:02:15.600
any member of it and scale it
up or down, it's still going

00:02:15.600 --> 00:02:16.340
to be an L.

00:02:16.340 --> 00:02:18.970
So this was a subspace
when we defined this.

00:02:18.970 --> 00:02:22.380
And just as a bit of a reminder
of what it was, we

00:02:22.380 --> 00:02:24.980
were able to figure out what
this projection is

00:02:24.980 --> 00:02:25.960
for some line L.

00:02:25.960 --> 00:02:31.060
If you have some spanning
vector, the projection onto

00:02:31.060 --> 00:02:36.620
this line L that goes through
the origin of the vector x, we

00:02:36.620 --> 00:02:43.330
figured out was x dot your
spanning vector for your line,

00:02:43.330 --> 00:02:48.070
so x dot v over v dot v,
which is really just

00:02:48.070 --> 00:02:50.290
the length of v squared.

00:02:50.290 --> 00:02:53.270
So all of this was a number and
you want it to be in the

00:02:53.270 --> 00:02:55.480
same direction as your line.

00:02:55.480 --> 00:02:57.525
It's going to be another
vector in your line.

00:02:57.525 --> 00:03:00.930
So it's going to be times
the vector v.

00:03:00.930 --> 00:03:03.800
So it's just going to be a
scaled up or scaled down

00:03:03.800 --> 00:03:05.720
version of your spanning
vector.

00:03:05.720 --> 00:03:07.890
Maybe your spanning vector
is like that.

00:03:07.890 --> 00:03:09.470
And really any vector
in your line could

00:03:09.470 --> 00:03:11.280
be a spanning vector.

00:03:11.280 --> 00:03:14.440
Any vector other than
the zero vector.

00:03:14.440 --> 00:03:18.340
Now that was a projection onto
a line which was a special

00:03:18.340 --> 00:03:19.940
kind of subspace.

00:03:19.940 --> 00:03:23.390
But now we're going to broaden
our definition of a projection

00:03:23.390 --> 00:03:25.480
to any subspace.

00:03:25.480 --> 00:03:30.180
So we already know that if-- let
me draw a little dividing

00:03:30.180 --> 00:03:33.540
line to show that we're doing
something slightly different--

00:03:33.540 --> 00:03:51.040
if v is a subspace of Rn then
v complement is also a

00:03:51.040 --> 00:03:53.130
subspace space of Rn.

00:03:53.130 --> 00:04:00.550
So the orthogonal complement
of v is also a subspace.

00:04:00.550 --> 00:04:02.770
And let's say we have some
members, or let me

00:04:02.770 --> 00:04:03.870
write it this way.

00:04:03.870 --> 00:04:05.740
If we have these two subspaces--
you have a

00:04:05.740 --> 00:04:08.160
subspace and you have this
orthogonal complement-- we

00:04:08.160 --> 00:04:11.670
already learned that if you
have any member of Rn-- so

00:04:11.670 --> 00:04:17.079
let's say that x is a member
of our Rn-- then x can be

00:04:17.079 --> 00:04:25.170
represented as a sum of a member
of v and a member of

00:04:25.170 --> 00:04:27.710
the orthogonal complement
of v.

00:04:27.710 --> 00:04:32.490
Where-- let me write this-- the
vector v is a member of

00:04:32.490 --> 00:04:38.670
the subspace v and the vector
w is a member of the

00:04:38.670 --> 00:04:42.780
orthogonal complement
of the subspace v.

00:04:42.780 --> 00:04:43.450
Just like that.

00:04:43.450 --> 00:04:45.390
We saw this several
videos ago.

00:04:45.390 --> 00:04:49.500
We proved that this was true
for any member Rn.

00:04:49.500 --> 00:04:57.650
Now given that, we can define
the projection of x onto the

00:04:57.650 --> 00:05:05.930
subspace v as being equal to,
just the part of x -- these

00:05:05.930 --> 00:05:09.530
are two orthogonal parts of x--
we define the projection

00:05:09.530 --> 00:05:12.500
onto v as a part of x
that came from v.

00:05:12.500 --> 00:05:15.730
It's equal to just
that vector v.

00:05:15.730 --> 00:05:22.710
Alternately you could say that
the projection of x onto the

00:05:22.710 --> 00:05:26.260
orthogonal complement of-- sorry
I wrote transpose-- the

00:05:26.260 --> 00:05:30.960
orthogonal complement of v is
going to be equal to w.

00:05:30.960 --> 00:05:33.590
So this piece right here
is a projection onto

00:05:33.590 --> 00:05:34.980
the subspace v.

00:05:34.980 --> 00:05:38.920
This piece right here is a
projection onto the orthogonal

00:05:38.920 --> 00:05:41.710
complement of the subspace v.

00:05:41.710 --> 00:05:45.000
Now what I want to do in this
video is show you that these

00:05:45.000 --> 00:05:49.060
two definitions-- that this
definition right here which is

00:05:49.060 --> 00:05:51.490
then in conjunction with this
right here-- this is the

00:05:51.490 --> 00:05:55.800
equivalent to what we learned up
here if the subspace v that

00:05:55.800 --> 00:06:00.450
we're dealing with is a line.

00:06:00.450 --> 00:06:02.000
Because this was a
valid subspace.

00:06:02.000 --> 00:06:05.950
But not all subspaces are
going to be lines.

00:06:05.950 --> 00:06:10.490
And to see this we can revisit
an example that we saw several

00:06:10.490 --> 00:06:12.050
videos ago.

00:06:12.050 --> 00:06:16.140
Several videos ago we had
this matrix here A.

00:06:16.140 --> 00:06:17.580
This 2 by 2 matrix.

00:06:17.580 --> 00:06:20.100
And then we had this other
vector b that was a member of

00:06:20.100 --> 00:06:22.770
the column space of A.

00:06:22.770 --> 00:06:27.810
We did this problem to show you
that the shortest solution

00:06:27.810 --> 00:06:31.880
to this right here was a unique

00:06:31.880 --> 00:06:33.030
member of the row space.

00:06:33.030 --> 00:06:36.600
Hopefully that gets your memory
on track for this

00:06:36.600 --> 00:06:39.050
problem when we first did it.

00:06:39.050 --> 00:06:41.070
But let me graph it and show you
that for the solution of

00:06:41.070 --> 00:06:43.110
that problem we could have
just as easily taken a

00:06:43.110 --> 00:06:45.240
projection onto a subspace.

00:06:45.240 --> 00:06:47.870
Let me graph everything
in this problem.

00:06:47.870 --> 00:06:50.885
This might help you remember
also about the problem.

00:06:50.885 --> 00:06:54.370
So let me draw my axes
just like that.

00:06:54.370 --> 00:06:56.060
So the first thing we learned--
you know you could

00:06:56.060 --> 00:06:57.785
solve this but I already
did this in a video.

00:06:57.785 --> 00:07:00.135
I think it was two or three
videos ago-- the null space of

00:07:00.135 --> 00:07:04.090
A, or all of the x's that
satisfy Ax is equal to zero,

00:07:04.090 --> 00:07:07.630
is a span of the vector 2, 3.

00:07:07.630 --> 00:07:09.230
So you go 2 to the right.

00:07:09.230 --> 00:07:10.900
1, 2.

00:07:10.900 --> 00:07:13.040
And then you go 3 up.

00:07:13.040 --> 00:07:15.600
1, 2, 3.

00:07:15.600 --> 00:07:18.590
And so it's the span
of this vector.

00:07:18.590 --> 00:07:22.430
And so the span of that vector
is just all the points.

00:07:22.430 --> 00:07:24.530
Well that vector specifies
that point.

00:07:24.530 --> 00:07:26.570
But if you scale this vector
up and down you're going to

00:07:26.570 --> 00:07:28.850
specify all of the point
on this line.

00:07:28.850 --> 00:07:30.240
All the points on that line.

00:07:30.240 --> 00:07:33.210
Let me draw it like that.

00:07:33.210 --> 00:07:33.940
That's good enough.

00:07:33.940 --> 00:07:35.750
It shouldn't curve down
like that at the end.

00:07:35.750 --> 00:07:37.480
So let me draw that a
little straighter.

00:07:37.480 --> 00:07:39.860
So this is the null space.

00:07:39.860 --> 00:07:43.480
That is our null space of
that matrix right there.

00:07:43.480 --> 00:07:46.730
And then the row space was a
span of the vector 3, minus 2.

00:07:46.730 --> 00:07:47.830
You see that right here.

00:07:47.830 --> 00:07:49.660
3, minus 2 is the first row.

00:07:49.660 --> 00:07:51.350
This guy is just a multiple
of that one.

00:07:51.350 --> 00:07:53.020
That's why we don't have
this guy right here

00:07:53.020 --> 00:07:54.530
in the span as well.

00:07:54.530 --> 00:07:56.520
And if we were to graph
it, 3, minus 2.

00:07:56.520 --> 00:08:00.290
You go out 3, then
you go down 1, 2.

00:08:00.290 --> 00:08:04.360
it would be the span of this
vector right there.

00:08:04.360 --> 00:08:05.635
Let me draw it like that.

00:08:05.635 --> 00:08:08.930
Now you take all of the scalar
multiples of that vector and

00:08:08.930 --> 00:08:11.160
you put those vectors in
standard position.

00:08:11.160 --> 00:08:13.220
They're going to specify, or
their tips are going to be on

00:08:13.220 --> 00:08:16.535
points along this line
right there.

00:08:16.535 --> 00:08:18.520
Along that line right there.

00:08:18.520 --> 00:08:21.060
I'm trying to make sure I
draw them orthogonally.

00:08:21.060 --> 00:08:23.940
So this right here
is the row space.

00:08:23.940 --> 00:08:27.040
That right there is the row
space of A which is the same

00:08:27.040 --> 00:08:30.410
thing as a column space
of A transpose.

00:08:30.410 --> 00:08:32.929
And we know that these guys are
each other's orthogonal

00:08:32.929 --> 00:08:33.880
complements.

00:08:33.880 --> 00:08:37.150
We know, we've seen this in
multiple videos, that the null

00:08:37.150 --> 00:08:43.580
space of A is the orthogonal
complement of the row space.

00:08:43.580 --> 00:08:46.990
And we also know that the
orthogonal complement of the

00:08:46.990 --> 00:08:50.280
null space is equal
to the row space.

00:08:50.280 --> 00:08:51.830
Everything in this
is orthogonal to

00:08:51.830 --> 00:08:52.910
everything in that.

00:08:52.910 --> 00:08:54.470
Everything in that
is orthogonal to

00:08:54.470 --> 00:08:55.340
everything in this.

00:08:55.340 --> 00:08:57.310
You can see it here
in this graph.

00:08:57.310 --> 00:09:00.720
That these two spaces, which are
represented by these lines

00:09:00.720 --> 00:09:03.140
that go through the origin,
are orthogonal.

00:09:03.140 --> 00:09:06.250
And it makes sense that any--
we said at really the

00:09:06.250 --> 00:09:08.930
beginning of the video-- that
anything in R2 in this

00:09:08.930 --> 00:09:16.860
situation, can be represented as
some sum of a unique member

00:09:16.860 --> 00:09:19.470
of our row space and a unique
member of its orthogonal

00:09:19.470 --> 00:09:19.970
complement.

00:09:19.970 --> 00:09:24.330
Let's say I have that
point right there.

00:09:24.330 --> 00:09:26.740
How could I represent it as a
sum of a member of this and a

00:09:26.740 --> 00:09:28.010
member of that?

00:09:28.010 --> 00:09:33.120
Well if I go along this guy, I
have this vector right here.

00:09:33.120 --> 00:09:37.380
I have that vector right
there along that line.

00:09:37.380 --> 00:09:41.860
And then I have this
vector right here.

00:09:41.860 --> 00:09:44.780
If I were to shift it-- this is
drawn in standard position,

00:09:44.780 --> 00:09:46.530
but I can draw a vector
wherever I want.

00:09:46.530 --> 00:09:48.795
These lines are just all of the
vectors drawn in standard

00:09:48.795 --> 00:09:50.780
position with their tails
at the origin.

00:09:50.780 --> 00:09:52.910
But we learned, in really, I
think, the first or second

00:09:52.910 --> 00:09:55.030
vector videos, that I can draw
them wherever I want.

00:09:55.030 --> 00:09:57.500
So if I add this vector and that
vector, I can shift this

00:09:57.500 --> 00:10:00.840
vector over and this vector
will be right there.

00:10:00.840 --> 00:10:01.550
And there you have it.

00:10:01.550 --> 00:10:04.690
I took an arbitrary point in R2
and I can represent it as a

00:10:04.690 --> 00:10:07.900
sum of a member of my row space
and a member of the row

00:10:07.900 --> 00:10:11.570
space's orthogonal complement
or the null space.

00:10:11.570 --> 00:10:13.740
But just to review, what we
originally did in that problem

00:10:13.740 --> 00:10:15.910
is we looked at the solution
set of this.

00:10:15.910 --> 00:10:19.190
We said the solution set of
this looks like this.

00:10:19.190 --> 00:10:23.000
It has a particular solution
plus members of your null

00:10:23.000 --> 00:10:24.820
space, plus homogeneous
solutions.

00:10:24.820 --> 00:10:26.870
We've seen that multiple
videos ago.

00:10:26.870 --> 00:10:30.190
So 3, 0-- it looks
like this-- plus

00:10:30.190 --> 00:10:31.540
members of the null space.

00:10:31.540 --> 00:10:34.690
So your solution set is going
to be parallel to this but

00:10:34.690 --> 00:10:36.970
shifted to the right by 3.

00:10:36.970 --> 00:10:42.010
So it looks-- let me draw it a
little neater than that-- Let

00:10:42.010 --> 00:10:44.040
me draw it like that.

00:10:44.040 --> 00:10:46.960
And then it goes down like,
the second part I didn't

00:10:46.960 --> 00:10:48.630
draw-- there you go.

00:10:48.630 --> 00:10:49.840
Oh that's not good either.

00:10:49.840 --> 00:10:51.620
Maybe I'm being too picky.

00:10:51.620 --> 00:10:53.635
OK, so this is your
solution set.

00:10:58.480 --> 00:11:00.740
And if you remember in that
video we said, hey there's

00:11:00.740 --> 00:11:04.040
some member of this solution set
that is also a member of

00:11:04.040 --> 00:11:07.280
our row space and that member of
the solution set that is a

00:11:07.280 --> 00:11:08.560
member of our row space
is going to be

00:11:08.560 --> 00:11:10.360
the shortest solution.

00:11:10.360 --> 00:11:11.170
And we saw that.

00:11:11.170 --> 00:11:12.910
You can see it visually
right here.

00:11:12.910 --> 00:11:13.210
Right?

00:11:13.210 --> 00:11:15.730
This vector right here.

00:11:15.730 --> 00:11:17.160
It is in our row space.

00:11:17.160 --> 00:11:18.990
It is a member of
our row space.

00:11:18.990 --> 00:11:22.510
And it also specifies a point
on our solution set.

00:11:22.510 --> 00:11:23.950
And you could see visually
that it's going to be the

00:11:23.950 --> 00:11:25.610
shortest solution.

00:11:25.610 --> 00:11:27.930
And one way you could think
about it is, this is the

00:11:27.930 --> 00:11:35.470
projection-- let me pick a good,
different, new color--

00:11:35.470 --> 00:11:40.140
any solution on our solution
set-- let me see right there--

00:11:40.140 --> 00:11:42.370
let's say that that is some
arbitrary solution on our

00:11:42.370 --> 00:11:43.620
solution set.

00:11:43.620 --> 00:11:44.210
Right?

00:11:44.210 --> 00:11:47.230
That's going to be a point in R2
and any point in R2 can be

00:11:47.230 --> 00:11:51.320
represented as a sum of some
vector in our row space and

00:11:51.320 --> 00:11:53.420
some vector in our null space.

00:11:53.420 --> 00:11:56.560
And so if I have this vector
right here, how can I do that?

00:11:56.560 --> 00:11:59.980
Well, I could represent it as
a sum of this guy right here

00:11:59.980 --> 00:12:03.340
and then this vector
right here.

00:12:03.340 --> 00:12:05.180
That vector right here.

00:12:05.180 --> 00:12:07.350
And this vector right
here is clearly a

00:12:07.350 --> 00:12:08.550
member of my null space.

00:12:08.550 --> 00:12:09.540
I just shifted over.

00:12:09.540 --> 00:12:12.040
This line is only when I draw
in standard position.

00:12:12.040 --> 00:12:14.670
This vector right here-- I'm
just showing it heads to

00:12:14.670 --> 00:12:17.430
tails-- if I add this member of
my row space to this member

00:12:17.430 --> 00:12:20.970
of my null space, I get an
arbitrary solution to my

00:12:20.970 --> 00:12:22.070
solution set.

00:12:22.070 --> 00:12:25.330
And if you think about it, the
projection of my arbitrary

00:12:25.330 --> 00:12:32.350
solution onto my row space will
be this guy right here.

00:12:32.350 --> 00:12:35.680
And that just comes from our--
well there are two ways to

00:12:35.680 --> 00:12:39.030
think about it-- we could say
that this is the solution

00:12:39.030 --> 00:12:40.110
right here.

00:12:40.110 --> 00:12:43.480
We could say our solution right
here is equal to some

00:12:43.480 --> 00:12:47.595
member of my row space plus some
member of my null space.

00:12:47.595 --> 00:12:48.820
This is the row space.

00:12:48.820 --> 00:12:50.370
That is the null space.

00:12:50.370 --> 00:12:53.050
And so by the definition of a
projection onto a subspace I

00:12:53.050 --> 00:12:57.740
just gave you, we know that
the projection of this

00:12:57.740 --> 00:13:03.510
solution onto my-- let me write
a little bit-- onto my

00:13:03.510 --> 00:13:09.260
row space of my solution,
is just equal

00:13:09.260 --> 00:13:10.330
to this first thing.

00:13:10.330 --> 00:13:13.640
It's equal to the component of
it that's in my row space.

00:13:13.640 --> 00:13:16.800
It's other component, we could
call it, is in the orthogonal

00:13:16.800 --> 00:13:18.290
complement of my row space.

00:13:18.290 --> 00:13:19.310
Or it's in my null space.

00:13:19.310 --> 00:13:22.600
So this is just going to be
equal to the R vector.

00:13:22.600 --> 00:13:27.220
Now, I want to show you that
that is essentially equal to

00:13:27.220 --> 00:13:29.760
the definition that
we did before.

00:13:29.760 --> 00:13:32.100
That this is completely
identical to the definition of

00:13:32.100 --> 00:13:34.750
a projection onto a line because
in this case the

00:13:34.750 --> 00:13:36.900
subspace is a line.

00:13:36.900 --> 00:13:40.090
So let's find a solution set.

00:13:40.090 --> 00:13:42.820
And the easiest one, the easiest
solution that we could

00:13:42.820 --> 00:13:45.370
find is if we set C as
equal to 0 here.

00:13:45.370 --> 00:13:48.973
We know that x equals 3, 0 is
one of these solutions.

00:13:48.973 --> 00:13:52.000
So x equals 3, 0 looks
like that.

00:13:52.000 --> 00:13:59.560
So we know x equals 3,
0 is a solution.

00:13:59.560 --> 00:14:01.020
And what we want to do
is we want to find

00:14:01.020 --> 00:14:02.530
the shortest solution.

00:14:02.530 --> 00:14:05.450
Or we want to find the
projection of x

00:14:05.450 --> 00:14:07.490
onto the row space.

00:14:07.490 --> 00:14:10.930
Or if we wanted, we could also
think of it is a projection of

00:14:10.930 --> 00:14:12.190
x onto this line.

00:14:12.190 --> 00:14:15.240
This line is equal
to the row space.

00:14:15.240 --> 00:14:16.130
So let's do that.

00:14:16.130 --> 00:14:18.580
And I'm doing this to show you
that this definition of a

00:14:18.580 --> 00:14:21.790
projection onto a subspace that
I've just introduced you

00:14:21.790 --> 00:14:25.790
to in this video, it is
completely identical to the

00:14:25.790 --> 00:14:28.500
definition, or it's not
identical, it's consistent

00:14:28.500 --> 00:14:30.660
with the definition of a
projection onto a line.

00:14:30.660 --> 00:14:33.400
Although this is more general
because a subspace doesn't

00:14:33.400 --> 00:14:34.150
have to be a line.

00:14:34.150 --> 00:14:35.270
But in this case it is a line.

00:14:35.270 --> 00:14:36.350
So let's do that.

00:14:36.350 --> 00:14:45.930
So the projection of the vector
3, 0 onto our row

00:14:45.930 --> 00:14:50.610
space, which is a line so we
can use that formula, it is

00:14:50.610 --> 00:14:56.320
equal to 3, 0 dot the
spanning vector for

00:14:56.320 --> 00:14:57.950
our row space, right?

00:14:57.950 --> 00:14:59.780
Dot the spanning vector
for our row space.

00:14:59.780 --> 00:15:01.600
So it's 3, minus 2.

00:15:01.600 --> 00:15:03.620
There's a bunch of spanning
vectors for your row space.

00:15:03.620 --> 00:15:05.460
This is just the one we
happened to pick.

00:15:05.460 --> 00:15:12.160
So dot 3, minus 2 all
over the spanning

00:15:12.160 --> 00:15:13.920
vector dotted with itself.

00:15:13.920 --> 00:15:17.410
3, minus 2 dot 3, minus 2.

00:15:17.410 --> 00:15:19.770
And then this is just going to
be one big scalar and then we

00:15:19.770 --> 00:15:23.220
want to multiply that-- or
essentially scale up-- our

00:15:23.220 --> 00:15:25.140
actual spanning vector
by that.

00:15:25.140 --> 00:15:29.070
So this is a projection of
this solution onto my row

00:15:29.070 --> 00:15:32.010
space, which should give me
this vector right here.

00:15:32.010 --> 00:15:34.250
Because we're just taking a
projection onto a line,

00:15:34.250 --> 00:15:37.780
because a row space in this
subspace is a line.

00:15:37.780 --> 00:15:41.380
And so we used the linear
projections that we first got

00:15:41.380 --> 00:15:43.170
introduced to, I think, when I
first started doing linear

00:15:43.170 --> 00:15:44.560
transformations.

00:15:44.560 --> 00:15:53.640
So let's see this is 3 times
3 plus 0 times minus 2.

00:15:53.640 --> 00:15:56.230
This right here is equal to 9.

00:15:56.230 --> 00:16:03.090
This is 3 times 3 plus minus
2 times minus 2.

00:16:03.090 --> 00:16:04.840
So that's 9 plus 4.

00:16:04.840 --> 00:16:06.180
That's 13.

00:16:06.180 --> 00:16:09.960
So it's 9/13 times this
vector right here.

00:16:09.960 --> 00:16:14.160
So it's going to be equal
to 9/13 times the

00:16:14.160 --> 00:16:16.230
vector 3, minus 2.

00:16:16.230 --> 00:16:24.590
Which is equal to the vector
27/13 and then minus 18/13,

00:16:24.590 --> 00:16:26.830
which is this vector
right here.

00:16:29.600 --> 00:16:31.910
We got this exact answer when
we first did it, although we

00:16:31.910 --> 00:16:34.230
just didn't use the projection
onto a line.

00:16:34.230 --> 00:16:36.410
But now we see that this is
exactly consistent with what

00:16:36.410 --> 00:16:37.500
we did before.

00:16:37.500 --> 00:16:39.480
We just used the projection
onto a line.

00:16:39.480 --> 00:16:42.560
And we see that this is
consistent with our new,

00:16:42.560 --> 00:16:44.655
broader definition
of a projection.

00:16:47.510 --> 00:16:50.040
Here we were able to do it
because we did it onto a line.

00:16:50.040 --> 00:16:56.350
But here I'm calling a
projection onto any subspace.

00:16:56.350 --> 00:16:58.800
We know how to do it if it's a
line, but so far I've just

00:16:58.800 --> 00:17:01.310
kind of defined it onto
an arbitrary subspace.

00:17:01.310 --> 00:17:04.300
But I haven't giving you a nice
mathematical, I guess, or

00:17:04.300 --> 00:17:07.280
computational way to figure out
what this is going to be

00:17:07.280 --> 00:17:09.010
if this isn't a line.

00:17:09.010 --> 00:17:12.140
In fact, I haven't even shown
you when this is general,

00:17:12.140 --> 00:17:14.770
whether this is definitely
a linear transformation.

00:17:14.770 --> 00:17:16.849
We know that when you take the
projection onto the line it's

00:17:16.849 --> 00:17:17.760
a linear transformation.

00:17:17.760 --> 00:17:19.810
But I haven't shown you that
when we take a projection onto

00:17:19.810 --> 00:17:23.490
an arbitrary subspace that it
is a linear projection.

00:17:23.490 --> 00:17:25.630
I'll do that in the
next video.


Showing that A-transpose x A is invertible

WEBVTT
Kind: captions
Language: en

00:00:00.590 --> 00:00:00.710
OK.

00:00:00.710 --> 00:00:02.870
I've got some matrix A.

00:00:02.870 --> 00:00:05.980
It's an n by k matrix.

00:00:05.980 --> 00:00:09.320
Let's say it's not just
any n by k matrix.

00:00:09.320 --> 00:00:13.650
This matrix A has a bunch of
columns that are all linearly

00:00:13.650 --> 00:00:14.400
independent.

00:00:14.400 --> 00:00:16.320
So, a1.

00:00:16.320 --> 00:00:23.440
a2, all the way through ak
are linearly independent.

00:00:23.440 --> 00:00:25.590
They are linearly independent
columns.

00:00:25.590 --> 00:00:27.250
Let me write that down.

00:00:27.250 --> 00:00:31.620
a1, a2, all the column
vectors of A.

00:00:31.620 --> 00:00:36.430
All the way through ak are
linearly independent.

00:00:36.430 --> 00:00:37.460
Now, what does that mean?

00:00:37.460 --> 00:00:42.250
That means that the only
solution to x1 times a1 plus

00:00:42.250 --> 00:00:49.820
x2 times a2, plus all the
way to xk times ak.

00:00:49.820 --> 00:00:54.320
The only solution to this is all
of these x's have to be 0.

00:00:54.320 --> 00:00:59.210
So, all xi's must
be equal to 0.

00:00:59.210 --> 00:01:01.740
That's what linear independence
implies.

00:01:01.740 --> 00:01:05.060
Or another way to write it is
all the solutions to this

00:01:05.060 --> 00:01:12.350
equation x1, x2, all
the way down to xk

00:01:12.350 --> 00:01:14.090
equaling the zero vector.

00:01:14.090 --> 00:01:17.410
That all the solutions to this
are all of these entries have

00:01:17.410 --> 00:01:18.260
to be equal to 0.

00:01:18.260 --> 00:01:21.730
This is just another way of
writing this right there.

00:01:21.730 --> 00:01:22.970
We've seen it multiple times.

00:01:22.970 --> 00:01:25.260
That's the zero vector
right there.

00:01:25.260 --> 00:01:28.850
So if all of these have to be 0,
that's like saying that the

00:01:28.850 --> 00:01:40.310
only solution to ax is equal
to 0, is x is equal to the

00:01:40.310 --> 00:01:41.320
zero vector.

00:01:41.320 --> 00:01:44.420
Or another way to say it-- this
is all coming out of the

00:01:44.420 --> 00:01:48.090
fact that this guy's columns
are linearly independent.

00:01:48.090 --> 00:01:50.430
So linear independence
of columns.

00:01:52.950 --> 00:01:55.870
Based on that, we can say, since
the only solution to ax

00:01:55.870 --> 00:01:58.630
is equal to 0 is x is equal to
the zero vector, we know that

00:01:58.630 --> 00:02:03.220
the null space of a must be
equal to the zero vector.

00:02:03.220 --> 00:02:05.900
Or it's a set with the just
the zero vector in it.

00:02:05.900 --> 00:02:08.280
And that is all a
bit of review.

00:02:08.280 --> 00:02:09.280
Now, n by k.

00:02:09.280 --> 00:02:10.350
We don't know its dimensions.

00:02:10.350 --> 00:02:12.600
It may or may not be
a square matrix.

00:02:12.600 --> 00:02:14.970
So we don't know, necessarily,
whether it's invertible and

00:02:14.970 --> 00:02:16.010
all of that.

00:02:16.010 --> 00:02:20.410
But maybe we can construct an
invertible matrix with it.

00:02:20.410 --> 00:02:24.530
So, let's study a transpose
times a.

00:02:24.530 --> 00:02:27.716
a transpose times a.

00:02:27.716 --> 00:02:30.610
A is an n by k matrix.

00:02:30.610 --> 00:02:34.030
A transpose will be
a k by n matrix.

00:02:34.030 --> 00:02:38.710
So, A transpose a is going
to be a k by k matrix.

00:02:38.710 --> 00:02:40.090
So it's a square matrix.

00:02:40.090 --> 00:02:44.260
So that's a nice place
to start for

00:02:44.260 --> 00:02:45.280
an invertible matrix.

00:02:45.280 --> 00:02:47.410
So let's see if it is
actually invertible.

00:02:47.410 --> 00:02:48.590
We don't know anything
about A.

00:02:48.590 --> 00:02:51.380
All we know is its columns
are linearly independent.

00:02:51.380 --> 00:02:53.760
Let's see if A transpose
a is invertible.

00:02:59.160 --> 00:03:01.980
Essentially, to show that it's
invertible, if we can show

00:03:01.980 --> 00:03:05.110
that all of its columns are
linearly independent, then

00:03:05.110 --> 00:03:06.220
we'll know it's invertible.

00:03:06.220 --> 00:03:09.020
If we have any-- and I'll
get back to this at

00:03:09.020 --> 00:03:09.710
the end of the video.

00:03:09.710 --> 00:03:20.400
But if you have a square
matrix with linearly

00:03:20.400 --> 00:03:24.670
independent columns-- remember,
the linearly

00:03:24.670 --> 00:03:30.030
independent columns all are
associated with pivot columns

00:03:30.030 --> 00:03:32.740
when you put them in reduced
row echelon form.

00:03:32.740 --> 00:03:34.660
So if you have a square matrix,
then you're going to

00:03:34.660 --> 00:03:38.070
have exactly-- so if it's a k by
k matrix, that means you're

00:03:38.070 --> 00:03:41.960
going to have k-- that means
that the reduced row echelon

00:03:41.960 --> 00:03:56.590
form of a matrix will have k
pivot columns and be k by k.

00:03:56.590 --> 00:03:58.740
And be a square k by k matrix.

00:03:58.740 --> 00:04:02.540
And there's only one k by k
matrix with k pivot columns.

00:04:02.540 --> 00:04:03.985
And that's the identity
matrix.

00:04:06.660 --> 00:04:09.240
The k by k identity matrix.

00:04:09.240 --> 00:04:12.460
And if when you do something
to reduce row echelon form,

00:04:12.460 --> 00:04:15.260
and it you got the identity
matrix, that means that your

00:04:15.260 --> 00:04:16.510
matrix is invertible.

00:04:21.019 --> 00:04:22.810
I could have probably left that
to the end of the video,

00:04:22.810 --> 00:04:23.420
but I just want to show you.

00:04:23.420 --> 00:04:26.610
If we can show that-- we already
know that this guy's

00:04:26.610 --> 00:04:29.670
square, that a transpose
A is a square matrix.

00:04:29.670 --> 00:04:32.780
If we can show that, given
that a has linearly

00:04:32.780 --> 00:04:37.110
independent columns, that a
transpose times A also has

00:04:37.110 --> 00:04:40.170
linearly independent columns,
and given the columns are

00:04:40.170 --> 00:04:42.330
linearly independent, and it's a
square matrix, that tells us

00:04:42.330 --> 00:04:44.390
that when we put it into reduced
row echelon form,

00:04:44.390 --> 00:04:45.920
we'll get the identity matrix.

00:04:45.920 --> 00:04:48.580
And that tells us that this
thing would be invertible.

00:04:48.580 --> 00:04:53.150
Let's see if we can prove that
all of this guy's columns are

00:04:53.150 --> 00:04:54.950
linearly independent.

00:04:54.950 --> 00:04:58.910
So let's say I have
some vector V.

00:04:58.910 --> 00:05:06.050
Let's say my vector V is a
member of the null space of a

00:05:06.050 --> 00:05:07.850
transpose A.

00:05:07.850 --> 00:05:15.230
That means that if I take a
transpose A times my vector v,

00:05:15.230 --> 00:05:18.060
I'm going to get the
zero vector.

00:05:18.060 --> 00:05:19.040
Fair enough?

00:05:19.040 --> 00:05:21.770
Now, what happens if I multiply
both sides of the

00:05:21.770 --> 00:05:25.220
equation times the transpose
of this guy?

00:05:25.220 --> 00:05:27.870
So I'll get a v transpose--
actually let me just do it

00:05:27.870 --> 00:05:28.990
right here.

00:05:28.990 --> 00:05:32.560
I multiply v transpose
on this side, and v

00:05:32.560 --> 00:05:35.250
transpose on this side.

00:05:35.250 --> 00:05:39.290
You could view this as a
matrix vector product.

00:05:39.290 --> 00:05:39.810
Right?

00:05:39.810 --> 00:05:45.990
Or, in general, if you take a
row vector times a column

00:05:45.990 --> 00:05:48.380
vector, it's essentially
their dot product.

00:05:48.380 --> 00:05:50.610
So this right-hand side of the
equation, you dot anything

00:05:50.610 --> 00:05:51.860
with the zero vector.

00:05:54.300 --> 00:05:57.000
That is just going to
be the zero vector.

00:05:57.000 --> 00:05:59.870
Now what is the left-hand side
of this going to be?

00:05:59.870 --> 00:06:01.960
We've seen this before.

00:06:01.960 --> 00:06:06.330
If you have the transpose of--
we can view this as, even

00:06:06.330 --> 00:06:09.180
though it's a transpose of a
vector, you can view it as a--

00:06:09.180 --> 00:06:10.520
it is a row factor,
but you could also

00:06:10.520 --> 00:06:12.730
view it as a matrix.

00:06:12.730 --> 00:06:15.220
Right?

00:06:15.220 --> 00:06:20.650
Let's say v is a
k by 1 matrix.

00:06:20.650 --> 00:06:23.450
v transpose will be
a 1 by k matrix.

00:06:23.450 --> 00:06:24.510
We've seen this before.

00:06:24.510 --> 00:06:30.120
That that is equal to the
reverse product, the transpose

00:06:30.120 --> 00:06:31.250
of the reverse product.

00:06:31.250 --> 00:06:34.340
Or if we take the product of two
things and transpose it,

00:06:34.340 --> 00:06:38.240
that's the same thing as taking
the reverse product of

00:06:38.240 --> 00:06:41.160
the transposes of either
of those two matrices.

00:06:41.160 --> 00:06:46.530
So given that, we can replace
this right here with a times a

00:06:46.530 --> 00:06:55.140
vector v transpose-- and we're
multiplying this vector times

00:06:55.140 --> 00:06:58.290
av times this vector
right here.

00:06:58.290 --> 00:07:02.070
And that is going to be equal
to the zero vector.

00:07:02.070 --> 00:07:04.160
Now, what is this?

00:07:04.160 --> 00:07:08.370
If I'm taking some vector's
transpose, and let's say this

00:07:08.370 --> 00:07:09.020
is a vector.

00:07:09.020 --> 00:07:11.760
Remember, even though I have a
matrix vector product right

00:07:11.760 --> 00:07:15.750
here, when I multiply a matrix
times this vector, it will

00:07:15.750 --> 00:07:17.260
result in another vector.

00:07:17.260 --> 00:07:19.840
So this is a vector, and this
is a vector right here.

00:07:19.840 --> 00:07:24.090
And if I take some vector and I
multiply its transpose times

00:07:24.090 --> 00:07:26.120
that vector-- we've
seen this before.

00:07:26.120 --> 00:07:29.970
That is the same thing
as y dot y.

00:07:29.970 --> 00:07:32.070
These two statements
are identical.

00:07:32.070 --> 00:07:41.770
So this thing right here is the
same thing as av dot av.

00:07:47.940 --> 00:07:50.050
And so what does the right-hand
side equal?

00:07:50.050 --> 00:07:52.920
The right-hand side is going
to be equal to 0.

00:07:52.920 --> 00:07:54.950
Actually let me just make
a correction up here.

00:07:54.950 --> 00:07:59.550
When I take v transpose times
the zero vector, v transpose

00:07:59.550 --> 00:08:01.300
is going to have k elements.

00:08:01.300 --> 00:08:03.730
And then the zero vector is also
going to have k elements.

00:08:03.730 --> 00:08:06.370
And when I take this product
that's like dotting it.

00:08:06.370 --> 00:08:08.540
You're taking the dot
product of v and 0.

00:08:08.540 --> 00:08:11.990
So this is a dot product of v
with the zero vector which is

00:08:11.990 --> 00:08:14.340
equal to zero, the
scalar zero.

00:08:14.340 --> 00:08:16.170
So this right here's
the scalar zero.

00:08:16.170 --> 00:08:17.690
I want to make sure
I clarify that.

00:08:17.690 --> 00:08:19.470
It wouldn't make sense
otherwise.

00:08:19.470 --> 00:08:22.340
So the right-hand side, when I
multiply the zero vector times

00:08:22.340 --> 00:08:25.710
the transpose of v, gets
just the number zero.

00:08:25.710 --> 00:08:27.620
No vector zero there.

00:08:27.620 --> 00:08:31.230
So this av dot av is going
to be equal to 0.

00:08:31.230 --> 00:08:38.490
Or we could say that the
magnitude, or the length, of

00:08:38.490 --> 00:08:40.450
av squared is equal to 0.

00:08:40.450 --> 00:08:43.400
Or that tells us that av
has to be equal to 0.

00:08:43.400 --> 00:08:47.740
The only vector whose length
is 0, is the zero vector.

00:08:47.740 --> 00:08:49.660
So av-- let me switch colors.

00:08:49.660 --> 00:08:52.240
Using that a little
bit too much.

00:08:52.240 --> 00:08:57.640
So we know that av
must be equal to

00:08:57.640 --> 00:09:01.080
0, to the zero vector.

00:09:01.080 --> 00:09:05.590
This must be equal to the zero
vector since its length is 0.

00:09:05.590 --> 00:09:08.310
Now, we started off with saying
v is a member of the

00:09:08.310 --> 00:09:12.340
null space of a transpose A.

00:09:12.340 --> 00:09:18.050
v can be any member of the null
space of a transpose A.

00:09:18.050 --> 00:09:22.650
But then from that assumption,
it turns out that V also has

00:09:22.650 --> 00:09:25.760
to be a member of the
null space of A.

00:09:25.760 --> 00:09:28.140
That av is equal to 0.

00:09:28.140 --> 00:09:30.270
Let's write that down.

00:09:30.270 --> 00:09:39.220
If v is a member of the null
space of a transpose A, then v

00:09:39.220 --> 00:09:42.420
is a member of the
null space of a.

00:09:42.420 --> 00:09:45.230
Now, our null space of A,
because A's columns are

00:09:45.230 --> 00:09:48.590
linearly independent, it only
contains one vector.

00:09:48.590 --> 00:09:51.300
It only contains the
zero vector.

00:09:51.300 --> 00:09:55.110
So, if this guy's a member of
the null space of A transpose

00:09:55.110 --> 00:09:57.750
A, and he has to be a member
of the null space of A,

00:09:57.750 --> 00:09:58.940
there's only one thing
he can be.

00:09:58.940 --> 00:10:00.730
There's only one entry there.

00:10:00.730 --> 00:10:06.670
So then v has to be equal
to the zero vector.

00:10:06.670 --> 00:10:10.760
Or another way to say that is,
any v that's in our null space

00:10:10.760 --> 00:10:13.820
of a transpose A has to
be the zero vector.

00:10:13.820 --> 00:10:19.150
Or the null space of a transpose
A is equal to the

00:10:19.150 --> 00:10:23.850
null space of a which is equal
to just the zero factor

00:10:23.850 --> 00:10:24.760
sitting there.

00:10:24.760 --> 00:10:26.910
Now, what does that do for us?

00:10:26.910 --> 00:10:35.530
That tells us that the only
solution to a transpose A

00:10:35.530 --> 00:10:39.640
times some vector x equal to
zero, this says that the only

00:10:39.640 --> 00:10:52.400
solution is the zero vector is
equal to the zero vector.

00:10:52.400 --> 00:10:52.700
Right?

00:10:52.700 --> 00:10:55.650
Because the null space of a
transpose A is the same as the

00:10:55.650 --> 00:10:56.560
null space of a.

00:10:56.560 --> 00:10:58.160
And that just has the
zero vector in it.

00:10:58.160 --> 00:11:00.920
The null space is just
the solution to this.

00:11:00.920 --> 00:11:05.190
So if the only solution to the
null space is this, that means

00:11:05.190 --> 00:11:14.570
that the columns of a transpose
A are linearly

00:11:14.570 --> 00:11:15.600
independent.

00:11:15.600 --> 00:11:19.350
You could, essentially, write
all of the linear combinations

00:11:19.350 --> 00:11:21.810
of the columns by the weights
of the entries of x.

00:11:21.810 --> 00:11:23.830
We actually did that
at the beginning.

00:11:23.830 --> 00:11:27.400
It's the same argument
we used up here.

00:11:27.400 --> 00:11:30.270
So if all of their columns are
linearly independent, and I

00:11:30.270 --> 00:11:33.530
said it over here, a transpose
A has linearly independent

00:11:33.530 --> 00:11:39.700
columns, and it's a square
matrix, that was from the

00:11:39.700 --> 00:11:41.160
definition of it.

00:11:41.160 --> 00:11:46.400
So we now know that A transpose
A if I were to put

00:11:46.400 --> 00:11:48.600
it-- let me do this way.

00:11:48.600 --> 00:11:51.640
That tells me that the reduced
row echelon form of a

00:11:51.640 --> 00:11:57.370
transpose A is going to be equal
to the k by k identity

00:11:57.370 --> 00:12:01.930
matrix which tells me that a
transpose A is invertible.

00:12:04.590 --> 00:12:05.900
Which is a pretty neat result.

00:12:05.900 --> 00:12:08.490
I started with the
matrix that has

00:12:08.490 --> 00:12:09.910
linearly independent columns.

00:12:09.910 --> 00:12:12.240
So it wasn't just any matrix.

00:12:12.240 --> 00:12:14.390
It wasn't just any run
of the mill matrix.

00:12:14.390 --> 00:12:16.680
It did have linearly independent
columns, but it

00:12:16.680 --> 00:12:17.870
might have weird dimensions.

00:12:17.870 --> 00:12:20.510
It's not necessarily
a square matrix.

00:12:20.510 --> 00:12:22.420
But I could construct
a square matrix.

00:12:22.420 --> 00:12:24.320
a transpose A with it.

00:12:24.320 --> 00:12:26.550
And we now know that
it also has

00:12:26.550 --> 00:12:27.890
linearly independent columns.

00:12:27.890 --> 00:12:29.410
It's a square matrix.

00:12:29.410 --> 00:12:31.700
And therefore it is invertible.


Rowspace solution to Ax = b example

WEBVTT
Kind: captions
Language: en

00:00:00.740 --> 00:00:03.970
I've got this 2 by 2 matrix A
here, and I've got this other

00:00:03.970 --> 00:00:06.490
member of R2, this vector b.

00:00:06.490 --> 00:00:09.310
Let's figure out all of the
interesting things that we can

00:00:09.310 --> 00:00:11.370
figure out about this matrix
and this vector.

00:00:11.370 --> 00:00:13.910
So the first thing of interest,
and this is all

00:00:13.910 --> 00:00:16.650
going to essentially help us
visualize what we learned in

00:00:16.650 --> 00:00:18.720
the last video, is the
null space of A.

00:00:21.340 --> 00:00:24.130
To figure out the null space
of A, we know that the null

00:00:24.130 --> 00:00:28.730
space of A is equal to the null
space of the reduced row

00:00:28.730 --> 00:00:30.525
echelon form of A.

00:00:30.525 --> 00:00:35.350
So let's just find out the
reduced row echelon form of A.

00:00:35.350 --> 00:00:38.830
Let's say we leave the
first row the same,

00:00:38.830 --> 00:00:41.060
we get 3 minus 2.

00:00:41.060 --> 00:00:43.600
And let's replace the second row
with the second row, minus

00:00:43.600 --> 00:00:47.750
2 times the first, so 6,
minus 2 times 3 is 0.

00:00:47.750 --> 00:00:52.630
And minus 4, minus 2
times minus 1 is 0.

00:00:52.630 --> 00:00:55.240
And then let's replace the first
row with the first row

00:00:55.240 --> 00:00:56.950
divided by 3.

00:00:56.950 --> 00:01:01.070
So then that becomes 1 minus
2/3, and the second

00:01:01.070 --> 00:01:02.420
row is still 0.

00:01:02.420 --> 00:01:05.700
So that's the reduced row
echelon form of A.

00:01:05.700 --> 00:01:06.820
And we want to find
its null space.

00:01:06.820 --> 00:01:09.150
So we want to find all of
the vectors that we

00:01:09.150 --> 00:01:10.610
can multiply it by.

00:01:10.610 --> 00:01:14.150
So this is the vector x1,
x2, that is equal to

00:01:14.150 --> 00:01:17.460
the 0 vector in R2.

00:01:17.460 --> 00:01:21.050
So the second row gives us no
information, 0 times x1, plus

00:01:21.050 --> 00:01:22.240
0 times x2 is equal to 0.

00:01:22.240 --> 00:01:23.240
No information there.

00:01:23.240 --> 00:01:25.180
So our only constraint
is the first row.

00:01:25.180 --> 00:01:31.710
1 times x1-- let me write it
here-- so 1 times x1, plus--

00:01:31.710 --> 00:01:38.950
let me write it this way--
minus 2/3 times x2 is

00:01:38.950 --> 00:01:41.160
equal to that 0.

00:01:41.160 --> 00:01:43.630
Is equal to that
0 right there.

00:01:43.630 --> 00:01:51.470
Or we could write that x1
is equal to 2/3 x2.

00:01:51.470 --> 00:01:59.090
So if we wanted to write the
null space of A-- actually,

00:01:59.090 --> 00:02:01.490
before I write that, just to
simplify things and just to

00:02:01.490 --> 00:02:06.030
show you that x2 isn't some
special number, let's just say

00:02:06.030 --> 00:02:12.600
that x2 is equal to t, where
t is some real number.

00:02:12.600 --> 00:02:19.620
And then we would have x1
is equal to 2/3 times t.

00:02:19.620 --> 00:02:24.130
So the null space of our matrix,
the null space of A,

00:02:24.130 --> 00:02:30.500
is going to be equal to the set
of all x1, x2's, that are

00:02:30.500 --> 00:02:36.480
equal to some real number
t times the vector.

00:02:39.990 --> 00:02:45.480
x2 is equal to t times 1, and
x1 is equal to 2/3 t.

00:02:45.480 --> 00:02:48.500
So 2/3, 1.

00:02:48.500 --> 00:02:49.520
Just like that.

00:02:49.520 --> 00:02:51.350
That is our null space.

00:02:51.350 --> 00:02:55.940
All of the multiples of
the vector 2/3, 1.

00:02:55.940 --> 00:02:58.910
And actually just to make this
a little bit simpler, we can

00:02:58.910 --> 00:03:02.300
pick c, let's pick t,
sorry, let's pick t

00:03:02.300 --> 00:03:04.620
to be equal to 3c.

00:03:04.620 --> 00:03:05.435
So then what do you have?

00:03:05.435 --> 00:03:09.870
If this is equal to 3c, if you
multiply the 3 times these, we

00:03:09.870 --> 00:03:16.500
could rewrite this as being
equal to the x1's, x2's that

00:03:16.500 --> 00:03:19.810
are equal to the scalar c, where
that is some other real

00:03:19.810 --> 00:03:28.230
number, times 3 times 2/3
is 2, 3 times 1 is 3.

00:03:28.230 --> 00:03:30.700
And the whole reason why I did
that is just to write the null

00:03:30.700 --> 00:03:34.100
space with a slightly simpler
basis vector.

00:03:34.100 --> 00:03:36.790
The one that didn't have
any fractions in it.

00:03:36.790 --> 00:03:38.490
Or we could also write that
the null space is

00:03:38.490 --> 00:03:40.470
equal to the span.

00:03:40.470 --> 00:03:44.980
The span of the vector 2, 3.

00:03:44.980 --> 00:03:46.580
All of these statements
are equivalent.

00:03:46.580 --> 00:03:48.230
So we figured out
the null space.

00:03:48.230 --> 00:03:50.800
Maybe another interesting thing,
especially if we're

00:03:50.800 --> 00:03:53.400
going to make this relate to
what we did in the last video,

00:03:53.400 --> 00:03:58.760
is find a solution set to the
equation Ax is equal to b.

00:03:58.760 --> 00:04:01.990
To do that, we just set up
an augmented matrix.

00:04:01.990 --> 00:04:07.970
So we set up an augmented
matrix, 3 minus 2, 6 minus 4,

00:04:07.970 --> 00:04:12.460
and we augment it
with b, 9, 18.

00:04:12.460 --> 00:04:15.290
And then we put the left-hand
side in reduced row echelon

00:04:15.290 --> 00:04:19.589
form, and we get-- let's keep
the first row the same-- we

00:04:19.589 --> 00:04:23.170
get 3 minus 2, and then 9.

00:04:23.170 --> 00:04:25.480
Let's replace the second row
with the second row minus 2

00:04:25.480 --> 00:04:31.690
times the first. So we get 6
minus 2 times 3, is 0, minus 4

00:04:31.690 --> 00:04:35.600
minus 2 times minus 2,
that's 0 as well.

00:04:35.600 --> 00:04:37.280
Minus 4 plus 4.

00:04:37.280 --> 00:04:42.380
18 minus 2 times 9, that's 18
minus 18, that's also 0.

00:04:42.380 --> 00:04:43.870
Almost there.

00:04:43.870 --> 00:04:45.660
Now let's just replace the first
row with the first row

00:04:45.660 --> 00:04:48.150
divided by 3.

00:04:48.150 --> 00:04:50.900
So the second row is going
to stay the same.

00:04:50.900 --> 00:04:57.800
And the first row is going to
be 1 minus 2/3, and then we

00:04:57.800 --> 00:05:02.300
get a 3 there.

00:05:02.300 --> 00:05:06.280
So if we wanted to rewrite this
is as an equation, we

00:05:06.280 --> 00:05:14.080
would write that the matrix 1
minus 2/3, 0, 0, times the

00:05:14.080 --> 00:05:19.420
vector that's in our solution
set, x1, x2, is equal to the

00:05:19.420 --> 00:05:21.920
vector 3, 0.

00:05:21.920 --> 00:05:24.560
Or another way to say it, the
second row gives us no real

00:05:24.560 --> 00:05:26.950
constraint, so we don't have
to pay attention to it.

00:05:26.950 --> 00:05:31.930
The first row tells us that
x1 minus 2/3, times

00:05:31.930 --> 00:05:36.120
x2 is equal to 3.

00:05:36.120 --> 00:05:43.120
Or, that x1 is equal
to 3, plus 2/3 x2.

00:05:43.120 --> 00:05:46.320
Let's do the same
exercise here.

00:05:46.320 --> 00:05:51.790
If we say that x2 is equal
to t, then x1 is

00:05:51.790 --> 00:05:55.570
equal to 3 plus 2/3t.

00:05:55.570 --> 00:06:00.140
Or we could say that the
solution set to Ax is equal to

00:06:00.140 --> 00:06:08.450
b is equal to the set of all x1,
x2's that are equal to--

00:06:08.450 --> 00:06:15.010
let's see, x1 is equal to 3 plus
2/3, so plus t times 2/3.

00:06:17.520 --> 00:06:18.770
Let's write it like that.

00:06:21.630 --> 00:06:23.300
So plus right there.

00:06:23.300 --> 00:06:27.900
And x2 is just equal to t, so
let's just go 1 times t plus,

00:06:27.900 --> 00:06:29.660
well let's just say 0.

00:06:29.660 --> 00:06:32.330
So that's our solution
set right there.

00:06:32.330 --> 00:06:35.060
And you might immediately
recognize that it's some

00:06:35.060 --> 00:06:39.185
particular solution plus some
scaled up version of our null

00:06:39.185 --> 00:06:40.490
space right there.

00:06:40.490 --> 00:06:41.690
That's our solution set.

00:06:41.690 --> 00:06:43.620
I could do that same
substitution

00:06:43.620 --> 00:06:44.950
that we did over here.

00:06:44.950 --> 00:06:50.990
I could say that let's replace
t with 3c, and so we could

00:06:50.990 --> 00:06:55.980
rewrite this as being equal to--
let's rewrite it neatly--

00:06:55.980 --> 00:07:02.750
x1, x2 is equal to the vector--
I'm going to run out

00:07:02.750 --> 00:07:07.140
of space-- 3, 0, plus c, times--
let me scroll over a

00:07:07.140 --> 00:07:14.390
little bit-- plus c, times the
vector, 3 times 2/3 is 2, 3

00:07:14.390 --> 00:07:15.640
times 1 is 3.

00:07:18.390 --> 00:07:22.365
So it's equal to the vector 3, 0
plus some member of our null

00:07:22.365 --> 00:07:23.060
space, right?

00:07:23.060 --> 00:07:25.410
The null space was the span
of 2, 3, or all the scalar

00:07:25.410 --> 00:07:26.380
multiples of our null space.

00:07:26.380 --> 00:07:29.060
And this is where c is
any real number.

00:07:29.060 --> 00:07:30.540
So that's our solution set.

00:07:30.540 --> 00:07:32.140
That's our null space.

00:07:32.140 --> 00:07:34.160
Now, one other thing that might
be of interest, just

00:07:34.160 --> 00:07:36.540
because it's the orthogonal
complement to the null space,

00:07:36.540 --> 00:07:38.540
and this relates to what we were
doing in the last video,

00:07:38.540 --> 00:07:41.970
is what the row space of A is.

00:07:41.970 --> 00:07:44.130
So the row space of
A is just the

00:07:44.130 --> 00:07:47.000
column space of A transpose.

00:07:47.000 --> 00:07:50.230
This is the row space.

00:07:50.230 --> 00:07:52.540
This is the row space of A.

00:07:52.540 --> 00:07:53.700
And what is this equal to?

00:07:53.700 --> 00:07:59.510
This is equal to the span
of the row vectors of A.

00:07:59.510 --> 00:08:09.560
So we have 3 minus 2, and
we have 6 minus 4.

00:08:12.490 --> 00:08:15.170
But this guy is just 2 times
that vector right there, so we

00:08:15.170 --> 00:08:15.950
can just ignore it.

00:08:15.950 --> 00:08:23.690
So it's just the span of
the vector 3 minus 2.

00:08:23.690 --> 00:08:24.950
Now let's graph all of these.

00:08:24.950 --> 00:08:28.200
So we have the row space, we
have the solution set and we

00:08:28.200 --> 00:08:30.040
have the null space.

00:08:30.040 --> 00:08:33.100
Let's see if we can graph it.

00:08:33.100 --> 00:08:37.190
We can make a reasonable graph
of this thing right here.

00:08:37.190 --> 00:08:42.309
So I have my vertical
axis right there.

00:08:42.309 --> 00:08:47.780
I have my horizontal
axis right there.

00:08:47.780 --> 00:08:50.820
Now what does the null
space look like?

00:08:50.820 --> 00:08:53.310
It's all the multiples
of 2, 3.

00:08:53.310 --> 00:08:59.800
So if I go out 1, 2 and then I
go up 1, 2, 3, the vector 2, 3

00:08:59.800 --> 00:09:02.970
looks like this in the
standard position.

00:09:02.970 --> 00:09:05.975
And the null space is all of
the multiples of this.

00:09:05.975 --> 00:09:07.890
So, all the multiples of that.

00:09:07.890 --> 00:09:10.290
So if you do all the multiples
of that, you get a line that

00:09:10.290 --> 00:09:12.156
looks something like this.

00:09:12.156 --> 00:09:15.400
If you do all the multiples of
that vector, you get a bunch

00:09:15.400 --> 00:09:17.740
of vectors that point to
every point on this

00:09:17.740 --> 00:09:18.910
green line right there.

00:09:18.910 --> 00:09:23.440
So that right there is
the null space of A.

00:09:23.440 --> 00:09:25.250
Now what is the solution set?

00:09:25.250 --> 00:09:28.610
The solution set is
the vector 3, 0.

00:09:28.610 --> 00:09:32.540
So it is the vector
3, 0, so 1, 2, 3.

00:09:32.540 --> 00:09:37.070
So it is the vector 3 , 0, plus

00:09:37.070 --> 00:09:38.430
members of the null space.

00:09:38.430 --> 00:09:41.090
So if you take any of these --
let's say if you add 2, 3-- 2,

00:09:41.090 --> 00:09:44.160
3 would look like this.

00:09:44.160 --> 00:09:46.100
But you're adding any
multiple of 2, 3.

00:09:46.100 --> 00:09:48.690
So when you add all the
different multiples of 2, 3,

00:09:48.690 --> 00:09:49.850
you get this line right here.

00:09:49.850 --> 00:09:54.020
It's shifted, essentially,
by that vector 3, 0.

00:09:54.020 --> 00:09:57.630
So this right here is
the solution set.

00:09:57.630 --> 00:10:04.320
So solution to Ax
is equal to b.

00:10:04.320 --> 00:10:06.350
That's the solution
set right there.

00:10:06.350 --> 00:10:07.510
Now, what's our row space?

00:10:07.510 --> 00:10:09.520
The row space is all of
the multiples of the

00:10:09.520 --> 00:10:11.700
vector 3 minus 2.

00:10:11.700 --> 00:10:13.610
So what does 3 minus
2 look like?

00:10:13.610 --> 00:10:17.940
The vector 3 minus 2, we go 1,
2, 3, and then we go down 2.

00:10:17.940 --> 00:10:24.150
So you go 1, 2, and it's going
to look like this.

00:10:24.150 --> 00:10:26.760
That's going to look something
like this, and then if you

00:10:26.760 --> 00:10:31.920
were to graph it and you can
see, well, it's going to look

00:10:31.920 --> 00:10:36.230
something like-- I want to do it
relatively neatly-- so it's

00:10:36.230 --> 00:10:38.280
going to look something
like this.

00:10:38.280 --> 00:10:41.420
Let me do it in another
color actually.

00:10:41.420 --> 00:10:45.700
It's going to look like
that right-- I can't

00:10:45.700 --> 00:10:49.090
do it at all straight.

00:10:49.090 --> 00:10:50.850
Oh, this is horrible.

00:10:50.850 --> 00:10:54.570
Something about the bottom right
hand side of my-- that's

00:10:54.570 --> 00:10:56.760
a pretty decent attempt.

00:10:56.760 --> 00:11:02.400
So that right there
is my row space.

00:11:02.400 --> 00:11:06.470
Because if you have 3 minus 2
you get roughly to that point,

00:11:06.470 --> 00:11:09.840
and then you want all
of the multiples of

00:11:09.840 --> 00:11:11.160
that vector, right?

00:11:11.160 --> 00:11:14.450
3 minus 2, if you multiplied it
by minus 1, you would get

00:11:14.450 --> 00:11:18.580
the vector minus 3, 2, so
it would look like this.

00:11:18.580 --> 00:11:20.160
It would be minus 3, 2.

00:11:20.160 --> 00:11:23.100
It would look like this, 1, 2,
3 and then up 2, it would be

00:11:23.100 --> 00:11:24.820
like that, so all of the scalar

00:11:24.820 --> 00:11:26.160
multiples of that as well.

00:11:26.160 --> 00:11:27.050
That is our column space,.

00:11:27.050 --> 00:11:30.050
And notice-- I'm sorry that is
our row space-- and notice

00:11:30.050 --> 00:11:33.940
that our row space is orthogonal
to our null space.

00:11:37.480 --> 00:11:40.830
Now this is a nice visual
representation of everything

00:11:40.830 --> 00:11:44.150
we can do essentially with this
matrix right here, which

00:11:44.150 --> 00:11:45.900
is our matrix A.

00:11:45.900 --> 00:11:49.170
But in the last video, we had
a very interesting result.

00:11:49.170 --> 00:11:51.310
We had a very interesting
result.

00:11:51.310 --> 00:11:55.120
We said that if we have some
b-- let me do it in a new

00:11:55.120 --> 00:11:58.840
color-- we found out that if we
had some b that is a member

00:11:58.840 --> 00:12:05.450
of the column space of A, then
the solution with the smallest

00:12:05.450 --> 00:12:10.930
length, or we can say the
smallest, or the shortest

00:12:10.930 --> 00:12:28.700
solution to Ax is equal to b is
a unique member of the row

00:12:28.700 --> 00:12:32.570
space of A.

00:12:32.570 --> 00:12:36.310
Now let's see, this is our big
takeaway from the last video.

00:12:36.310 --> 00:12:37.410
And it might have seemed
a little hard

00:12:37.410 --> 00:12:38.960
to visualize before.

00:12:38.960 --> 00:12:40.900
But now that we have graphed
it all out, I think we can

00:12:40.900 --> 00:12:42.110
visualize it.

00:12:42.110 --> 00:12:44.920
This blue line right here
is a solution set.

00:12:44.920 --> 00:12:49.510
The row space right here is this
line that's perpendicular

00:12:49.510 --> 00:12:50.340
to the solution set.

00:12:50.340 --> 00:12:57.600
But notice, one of the vectors
on it is both pointing to

00:12:57.600 --> 00:13:00.300
something, pointing to a
position on the solution set,

00:13:00.300 --> 00:13:02.110
and it is in my row space.

00:13:02.110 --> 00:13:05.760
This vector right here, that
I'm doing in a bold green,

00:13:05.760 --> 00:13:08.870
That vector right there, we
could call that vector r,

00:13:08.870 --> 00:13:13.220
because that vector is both a
member of my row space, right?

00:13:13.220 --> 00:13:16.990
It points-- if you do it as a
position vector, it points to

00:13:16.990 --> 00:13:20.300
a point on the line that
represents our row space.

00:13:20.300 --> 00:13:22.230
All of the members of our
row space point to

00:13:22.230 --> 00:13:23.990
points on this line.

00:13:23.990 --> 00:13:27.240
But at the same time, it also
points to a point right there

00:13:27.240 --> 00:13:29.440
that is a member of
our solution set.

00:13:29.440 --> 00:13:32.470
And notice, it is the only
vector in our row space that

00:13:32.470 --> 00:13:35.820
points to a point that is a
member of our solution set.

00:13:35.820 --> 00:13:38.120
And if you just look at it
from a geometric point of

00:13:38.120 --> 00:13:41.880
view, all of the other solutions
point to all of the

00:13:41.880 --> 00:13:42.860
other points on that line.

00:13:42.860 --> 00:13:47.490
So, that's a solution right
there, this is a solution

00:13:47.490 --> 00:13:49.850
right there, that is a
solution right there.

00:13:49.850 --> 00:13:54.300
All of the vectors that point
to things on this solution

00:13:54.300 --> 00:13:55.790
line right there, there's
all solutions.

00:13:55.790 --> 00:13:58.940
But the very shortest one
is this green one.

00:13:58.940 --> 00:14:01.540
This green one is essentially
orthogonal to it because it's

00:14:01.540 --> 00:14:03.080
a member of the row space.

00:14:03.080 --> 00:14:06.000
And this essentially has the
same slope as our null space.

00:14:06.000 --> 00:14:09.000
It's orthogonal, so it's kind of
the shortest path distance

00:14:09.000 --> 00:14:11.800
to getting to our
solution set.

00:14:11.800 --> 00:14:15.580
And for this exact example, we
can actually figure out what

00:14:15.580 --> 00:14:19.480
this shortest vector r is.

00:14:19.480 --> 00:14:22.280
We've got the vector 3,
0, over here, right?

00:14:22.280 --> 00:14:25.730
The vector 3, 0 is this
vector right here.

00:14:25.730 --> 00:14:29.690
So if you take 3, 0 minus your
vector-- so let me say r.

00:14:29.690 --> 00:14:31.010
Let me write this way.

00:14:31.010 --> 00:14:34.600
Our vector r, the special
shortest solution, it's going

00:14:34.600 --> 00:14:36.570
to be a member of our
column space.

00:14:36.570 --> 00:14:39.585
Our column space is a
span of 3 minus 2.

00:14:39.585 --> 00:14:42.180
So it's going to be equal
to some scalar

00:14:42.180 --> 00:14:46.440
multiple of 3 minus 2.

00:14:46.440 --> 00:14:52.570
Now, we know that 3, 0 points
to another solution on our

00:14:52.570 --> 00:14:53.750
solution set.

00:14:53.750 --> 00:14:55.970
But we know that if we take the
difference of these two

00:14:55.970 --> 00:15:03.080
vectors-- if I take 3, 0 and
from that I subtract the

00:15:03.080 --> 00:15:04.990
vector r, what do I get?

00:15:04.990 --> 00:15:06.740
I get this vector right there.

00:15:06.740 --> 00:15:08.690
Let me do that in
another color.

00:15:08.690 --> 00:15:12.310
I get this pink vector
right there.

00:15:12.310 --> 00:15:15.300
That pink vector, it's not in a
standard position, but that

00:15:15.300 --> 00:15:18.390
pink vector is going to be a
member of our null space.

00:15:18.390 --> 00:15:20.570
That pink vector is going to be
a member of our null space.

00:15:20.570 --> 00:15:22.130
So if I take 3, 0.

00:15:22.130 --> 00:15:30.770
That, minus my r, so minus c,
times 3 minus 2, right?

00:15:30.770 --> 00:15:33.550
That's going to be this pink
vector right there.

00:15:33.550 --> 00:15:36.480
So if I were to take the dot
product, so this vector right

00:15:36.480 --> 00:15:39.130
here-- let me make this clear;
that vector is this pink

00:15:39.130 --> 00:15:41.700
vector right there-- that's
in my null space.

00:15:41.700 --> 00:15:43.220
I could shift it over.

00:15:43.220 --> 00:15:44.700
It's not in standard position
there, but [? I'll have to ?]

00:15:44.700 --> 00:15:47.030
do it in the standard position,
it points to

00:15:47.030 --> 00:15:49.450
something on my null
space line.

00:15:49.450 --> 00:15:50.700
So that's in my null space.

00:15:50.700 --> 00:15:54.980
So if I were to dot it with any
member of my row space,

00:15:54.980 --> 00:15:56.350
it's going to be equal to 0.

00:15:56.350 --> 00:15:58.820
The row space is the orthogonal
complement of the

00:15:58.820 --> 00:15:59.890
null space.

00:15:59.890 --> 00:16:02.750
So let me take the dot product
of that, with some member of

00:16:02.750 --> 00:16:04.140
my row space.

00:16:04.140 --> 00:16:06.940
Might as well just take the
basis vector for my row space.

00:16:06.940 --> 00:16:08.780
That's a member of
my row space.

00:16:08.780 --> 00:16:11.990
So if I dot it with 3-- sorry, I
don't want you to write that

00:16:11.990 --> 00:16:14.430
as a fraction-- 3 minus 2.

00:16:14.430 --> 00:16:18.120
If I dot it with 3 minus 2,
that will be equal to 0.

00:16:18.120 --> 00:16:21.640
So let's see if we
can solve for c.

00:16:21.640 --> 00:16:25.770
So we get, let's see if we do
this, we get this inside part,

00:16:25.770 --> 00:16:27.610
this pink vector right
there is going to

00:16:27.610 --> 00:16:32.520
be 3 minus 3c, right?

00:16:32.520 --> 00:16:36.180
And then you're going to
have 0 minus minus 2c,

00:16:36.180 --> 00:16:37.430
so that's just 2c.

00:16:40.720 --> 00:16:43.720
So this part right here
simplifies to that.

00:16:43.720 --> 00:16:46.280
I just perform the scalar
multiplication and then the

00:16:46.280 --> 00:16:47.360
subtraction.

00:16:47.360 --> 00:16:51.530
And then if we dot that with
my basis vector for my row

00:16:51.530 --> 00:16:56.530
space, 3 minus 2, this
should be equal to 0.

00:16:56.530 --> 00:16:57.690
So what do we get?

00:16:57.690 --> 00:17:03.090
We get 3 times 3, which is 9,
minus-- let me see, 3 times

00:17:03.090 --> 00:17:10.140
3-- minus 3, or minus 3c times
minus 2-- let me write it this

00:17:10.140 --> 00:17:13.700
way-- plus-- let me
do it this way.

00:17:13.700 --> 00:17:15.950
This is probably the easiest
way to do it.

00:17:15.950 --> 00:17:17.500
Sorry, I was actually
doing it wrong.

00:17:17.500 --> 00:17:20.550
This is the first entry,
so you have 3 minus

00:17:20.550 --> 00:17:23.180
3c, times 3, right?

00:17:23.180 --> 00:17:26.359
3 minus 3c-- the first entry
times the first entry here--

00:17:26.359 --> 00:17:34.090
plus 2c, times minus
2, is equal to 0.

00:17:34.090 --> 00:17:36.930
I was performing that dot
product in a strange way.

00:17:36.930 --> 00:17:39.650
I think I got a little caught
off guard by these two terms.

00:17:39.650 --> 00:17:41.980
But anyway, the first term times
the first term, plus the

00:17:41.980 --> 00:17:44.250
second term times
the second term.

00:17:44.250 --> 00:17:45.610
That's what our dot
product is.

00:17:45.610 --> 00:17:46.510
This isn't the matrix.

00:17:46.510 --> 00:17:49.320
This is just a first and a
second term right there.

00:17:49.320 --> 00:17:51.330
So then what do we have here?

00:17:51.330 --> 00:18:03.430
We have 3 times 3 is 9, minus
9c, minus 4c, is equal to 0.

00:18:03.430 --> 00:18:07.190
This is 9 minus 13c
is equal to 0.

00:18:07.190 --> 00:18:12.870
Or we get 13, let's see, 9
is equal to 13c, or c is

00:18:12.870 --> 00:18:15.710
equal to 9 over 13.

00:18:15.710 --> 00:18:18.610
So just like that we've solved
for our special vector r.

00:18:18.610 --> 00:18:22.680
We said, hey look, if you take
this 3, 0 vector and the

00:18:22.680 --> 00:18:24.620
difference between that and our
vector r, you're going to

00:18:24.620 --> 00:18:28.620
get some vector, right here,
that is in our null space.

00:18:28.620 --> 00:18:32.340
And if you take the dot product
of that with some

00:18:32.340 --> 00:18:34.140
member in our row space,
you're going to get 0.

00:18:34.140 --> 00:18:36.500
And the dot product is that
entry times that entry, plus

00:18:36.500 --> 00:18:38.670
that entry times that entry.

00:18:38.670 --> 00:18:42.030
And so you get c is equal
to 9 over 13.

00:18:42.030 --> 00:18:47.700
So our special vector r, the
unique shortest solution to

00:18:47.700 --> 00:18:54.140
the equation Ax is equal to 0,
is 9 over 13 times the vector,

00:18:54.140 --> 00:19:00.170
times our basis vector 3 minus
2, or we could write that as

00:19:00.170 --> 00:19:03.160
27 over 13, right?

00:19:03.160 --> 00:19:09.530
And then we have 3 times--
and then 2 times 9 is

00:19:09.530 --> 00:19:13.090
minus 18 over 13.

00:19:13.090 --> 00:19:19.115
And that right there is our
shortest vector in our row

00:19:19.115 --> 00:19:21.520
space that satisfies
this equation.

00:19:21.520 --> 00:19:22.300
Let me write it better.

00:19:22.300 --> 00:19:27.230
It's the unique member of our
row space that also happens to

00:19:27.230 --> 00:19:30.980
be the shortest solution
to Ax is equal to b.

00:19:30.980 --> 00:19:33.430
So this is an example of what
we talked about in the last

00:19:33.430 --> 00:19:37.680
video, and hopefully you see
visually why this is so.