Invertible change of basis matrix

WEBVTT
Kind: captions
Language: en

00:00:00.640 --> 00:00:03.120
Like we've done in the last
several videos, let's assume

00:00:03.120 --> 00:00:07.170
that we have some set
of basis vectors B.

00:00:07.170 --> 00:00:11.640
And let's say our basis
is going to be v1 v2

00:00:11.640 --> 00:00:14.810
all the way to vk.

00:00:14.810 --> 00:00:18.340
So this will span a subspace
of dimension k.

00:00:18.340 --> 00:00:22.840
And let's assume that each of
these guys are members of Rn.

00:00:22.840 --> 00:00:27.400
So v1 v2 all the way to vk.

00:00:27.400 --> 00:00:30.530
They're all members of Rn.

00:00:30.530 --> 00:00:33.930
Now in the last video, we saw
that we can define a change of

00:00:33.930 --> 00:00:35.530
basis matrix.

00:00:35.530 --> 00:00:39.140
And it's a fancy word, but all
it means is a matrix that has

00:00:39.140 --> 00:00:41.720
these basis vectors
as its columns.

00:00:41.720 --> 00:00:48.970
So v1, v2 all the way to
vk as its columns.

00:00:48.970 --> 00:00:52.530
So we're going to have k columns
and we're going to

00:00:52.530 --> 00:00:53.530
have n rows.

00:00:53.530 --> 00:00:55.610
Because each of these guys are
members of Rn, so they're

00:00:55.610 --> 00:00:56.880
going to have n entries.

00:00:56.880 --> 00:00:58.840
So we're going to have n rows.

00:00:58.840 --> 00:01:01.220
So it's going to be
an n by k matrix.

00:01:01.220 --> 00:01:08.430
And we saw in the last video
that if I have some vector a

00:01:08.430 --> 00:01:14.030
that is a member of Rn-- and
assuming that a is in the span

00:01:14.030 --> 00:01:16.320
of B-- I can represent a.

00:01:16.320 --> 00:01:22.480
I could say that a is equal to
the change of basis matrix

00:01:22.480 --> 00:01:28.500
times the coordinates of a with
respect to our basis.

00:01:28.500 --> 00:01:30.550
This is what we saw
in the last video.

00:01:30.550 --> 00:01:34.030
If I have the coordinates of
a with respect to B, I can

00:01:34.030 --> 00:01:36.960
multiply it by the change of
basis matrix and I'll get my

00:01:36.960 --> 00:01:38.740
vector a in standard
coordinates.

00:01:38.740 --> 00:01:41.060
Or if I have my vector a in
standard coordinates, then I

00:01:41.060 --> 00:01:45.600
can solve for my vector a in
coordinates with respect to B.

00:01:45.600 --> 00:01:47.200
We saw that in the last video.

00:01:47.200 --> 00:01:50.490
Now let's take a special case.

00:01:50.490 --> 00:01:55.590
Let's assume that
C is invertible.

00:02:02.080 --> 00:02:03.530
What does that mean?

00:02:03.530 --> 00:02:05.880
Or what does that
tell us about C?

00:02:05.880 --> 00:02:07.430
Well if C is invertible,
it's two things.

00:02:07.430 --> 00:02:11.540
It means that C is a square
matrix or has the same number

00:02:11.540 --> 00:02:12.450
rows and columns.

00:02:12.450 --> 00:02:15.290
And that its rows or columns--
You can pick either of them--

00:02:15.290 --> 00:02:16.760
have to be linearly
independent.

00:02:16.760 --> 00:02:20.200
So linearly independent, let's
just pick columns.

00:02:20.200 --> 00:02:21.990
Now the second statement
is a bit redundant.

00:02:21.990 --> 00:02:24.950
We know that C has linearly
independent columns, because

00:02:24.950 --> 00:02:28.410
its columns are bases
for a subspace.

00:02:28.410 --> 00:02:31.750
So a basis, by definition, all
of the vectors have to be

00:02:31.750 --> 00:02:32.790
linearly independent.

00:02:32.790 --> 00:02:37.130
So we know this is
a bit redundant.

00:02:37.130 --> 00:02:39.280
But what's interesting is if we
know that C is invertible,

00:02:39.280 --> 00:02:41.200
C has to be square.

00:02:41.200 --> 00:02:44.760
And if all of these vectors are
members of Rn, then k has

00:02:44.760 --> 00:02:48.970
to be equal to n.

00:02:48.970 --> 00:02:55.600
So C is square means that k is
equal to n or that we have n

00:02:55.600 --> 00:02:56.850
basis vectors.

00:02:59.730 --> 00:03:03.520
Now if that's the case,
what is the span of B?

00:03:03.520 --> 00:03:04.430
Think about it.

00:03:04.430 --> 00:03:20.470
We have n linearly independent
vectors in Rn.

00:03:20.470 --> 00:03:23.920
So any time you have n linearly
independent vectors

00:03:23.920 --> 00:03:27.590
in Rn, those guys are
a basis for Rn.

00:03:27.590 --> 00:03:30.460
Because any basis that has n
entries-- and they're all

00:03:30.460 --> 00:03:35.110
linearly independent-- is going
to be a basis for Rn.

00:03:35.110 --> 00:03:42.640
So then B is a basis for Rn.

00:03:42.640 --> 00:03:45.870
So if we know that C is
invertible, we also know that

00:03:45.870 --> 00:03:50.570
you can get to any vector in Rn
by some linear combination

00:03:50.570 --> 00:03:54.820
of your basis vectors
right there.

00:03:54.820 --> 00:03:59.110
In the last video, we had to
make sure that this guy was in

00:03:59.110 --> 00:04:01.240
the span of these vectors.

00:04:01.240 --> 00:04:03.720
But now we don't have to make
sure, because if C is

00:04:03.720 --> 00:04:17.680
invertible, then the span of B
is going to be equal to Rn.

00:04:17.680 --> 00:04:19.899
Or another way you could say
it is if the span of B is

00:04:19.899 --> 00:04:21.110
equal to Rn.

00:04:21.110 --> 00:04:25.550
If we have n vectors here, if k
was equal to n, then we know

00:04:25.550 --> 00:04:28.040
that the span of B would
be equal to Rn.

00:04:28.040 --> 00:04:31.240
And so we'd have n vectors here,
n linearly independent

00:04:31.240 --> 00:04:34.295
columns here, and it would be
an n by n matrix with all of

00:04:34.295 --> 00:04:35.750
the columns linearly
independent.

00:04:35.750 --> 00:04:37.330
So then C would be invertible.

00:04:37.330 --> 00:04:39.590
So we could write
if and only if.

00:04:39.590 --> 00:04:41.280
And we could write
it the other way.

00:04:41.280 --> 00:04:53.530
If the span of B is Rn,
then C is invertible.

00:04:53.530 --> 00:04:56.350
And that's useful, because if
either of these things are

00:04:56.350 --> 00:05:00.040
true, then we can rewrite
the same equation.

00:05:00.040 --> 00:05:03.570
So let's say if we know this and
we're looking for that, we

00:05:03.570 --> 00:05:05.420
can just multiply
C times that.

00:05:05.420 --> 00:05:07.520
Let's say we know this and
we're looking for that.

00:05:07.520 --> 00:05:10.065
Before we had to do that
augmented matrix and solve for

00:05:10.065 --> 00:05:10.585
it, whatnot.

00:05:10.585 --> 00:05:15.650
But if we know C is invertible,
then one, we know

00:05:15.650 --> 00:05:19.020
that any vector here can
be represented in

00:05:19.020 --> 00:05:20.300
the span of our basis.

00:05:20.300 --> 00:05:22.240
So any vector here can be
represented as linear

00:05:22.240 --> 00:05:23.830
combinations of these guys.

00:05:23.830 --> 00:05:27.090
So you know that any vector can
be represented in these

00:05:27.090 --> 00:05:28.680
coordinates or with
coordinates with

00:05:28.680 --> 00:05:30.790
respect to our basis.

00:05:30.790 --> 00:05:34.140
We can multiply both sides of
this equation times C inverse.

00:05:34.140 --> 00:05:35.930
And what do you get
if you multiply?

00:05:35.930 --> 00:05:40.520
So it becomes C inverse C times
our coordinates of a

00:05:40.520 --> 00:05:45.030
with respect to B is equal
to C inverse times a.

00:05:45.030 --> 00:05:50.400
This is just the identity
matrix right there.

00:05:50.400 --> 00:05:54.220
Another way of writing this is
that the coordinates of a,

00:05:54.220 --> 00:05:59.100
with respect to our basis B,
which spans all of Rn, is

00:05:59.100 --> 00:06:04.270
equal to C inverse times
our vector a.

00:06:04.270 --> 00:06:06.230
Let's apply this a little bit.

00:06:06.230 --> 00:06:07.700
Let's apply this.

00:06:07.700 --> 00:06:11.050
Let's use this information, what
we've done in this video.

00:06:11.050 --> 00:06:13.780
Let's do some concrete
examples.

00:06:13.780 --> 00:06:16.420
So let's say I have
some basis.

00:06:20.480 --> 00:06:21.580
Let me define two vectors.

00:06:21.580 --> 00:06:22.290
I'll do it this way.

00:06:22.290 --> 00:06:28.310
So let's say I have v1 is equal
to the vector 1, 3.

00:06:28.310 --> 00:06:35.130
And let's say v2 is equal
to the vector 2, 1.

00:06:35.130 --> 00:06:43.110
And I have a basis that is equal
to the set of v1 and v2.

00:06:43.110 --> 00:06:45.750
Now I'll leave it for you to
verify that these guys are

00:06:45.750 --> 00:06:46.690
linearly independent.

00:06:46.690 --> 00:06:49.860
But if I have two linearly
independent vectors in R2,

00:06:49.860 --> 00:06:59.960
then B is a basis for R2.

00:06:59.960 --> 00:07:05.550
And if we write the change of
basis matrix, if we say C is

00:07:05.550 --> 00:07:10.740
equal to 1, 3, 2, 1, we know
that C is invertible.

00:07:10.740 --> 00:07:12.700
And actually to show that C
is invertible, we can just

00:07:12.700 --> 00:07:14.590
calculate its inverse.

00:07:14.590 --> 00:07:16.720
So what's the determinant
of C?

00:07:16.720 --> 00:07:24.760
The determinant of C is equal to
1 times 1 minus 2 times 3.

00:07:24.760 --> 00:07:27.340
So it's equal minus 5.

00:07:27.340 --> 00:07:28.880
That's the determinant of C.

00:07:28.880 --> 00:07:31.900
And so C inverse-- We figured
out a general formula for

00:07:31.900 --> 00:07:35.190
doing this for 2 by 2 matrices--
is equal to 1 over

00:07:35.190 --> 00:07:42.230
the determinant of C, so 1
over minus 5 times-- You

00:07:42.230 --> 00:07:45.510
switch these two guys, so you
switch the 1's, and you make

00:07:45.510 --> 00:07:47.380
these two guys negative.

00:07:47.380 --> 00:07:50.760
So minus 2, and then minus 3.

00:07:50.760 --> 00:07:54.890
And the very fact that this guy,
the determinant of C, was

00:07:54.890 --> 00:07:57.920
non-zero told us that
this was invertible.

00:07:57.920 --> 00:07:59.460
But anyway, this is C inverse.

00:07:59.460 --> 00:08:04.980
So let's say that I have
some vector a that

00:08:04.980 --> 00:08:06.910
is a member of R2.

00:08:09.480 --> 00:08:10.990
I'm just going to pick
some random numbers.

00:08:10.990 --> 00:08:13.600
Let's say that a is
equal to 7, 2.

00:08:13.600 --> 00:08:17.590
And I want to find out what the
coordinates of a are with

00:08:17.590 --> 00:08:20.900
respect to my basis B.

00:08:24.690 --> 00:08:26.980
Well we go to this situation.

00:08:26.980 --> 00:08:32.340
We know what a is, so we just
multiply a times C inverse to

00:08:32.340 --> 00:08:35.179
get this guy right here, to get
the coordinates of a with

00:08:35.179 --> 00:08:37.159
respect to B.

00:08:37.159 --> 00:08:38.340
So let me write that down.

00:08:38.340 --> 00:08:39.280
So what is C?

00:08:39.280 --> 00:08:40.419
So C is that.

00:08:40.419 --> 00:08:41.429
C inverse is that.

00:08:41.429 --> 00:08:46.370
So we could write the
coordinates of a with respect

00:08:46.370 --> 00:08:50.410
to B is equal to C inverse
times the standard

00:08:50.410 --> 00:08:51.990
coordinates of a.

00:08:51.990 --> 00:08:53.480
Or this is the same thing.

00:08:53.480 --> 00:08:54.680
Let me put the actual
numbers here.

00:08:54.680 --> 00:08:58.780
The coordinates of a with
respect to B are going to be

00:08:58.780 --> 00:09:05.900
equal to C inverse, which is
minus 1/5 times 1 minus 3

00:09:05.900 --> 00:09:15.030
minus 2, 1, times
a, times 7, 2.

00:09:15.030 --> 00:09:15.950
And what is this equal to?

00:09:15.950 --> 00:09:19.320
This is equal to minus 1/5.

00:09:19.320 --> 00:09:24.140
And then we're going to get 1
times 7 plus minus 2 times 2.

00:09:24.140 --> 00:09:25.180
So it's minus 4.

00:09:25.180 --> 00:09:27.270
So 7 minus 4 is 3.

00:09:27.270 --> 00:09:30.470
And then we're going to get
minus 3 times 7, which is

00:09:30.470 --> 00:09:33.660
minus 21, plus 1 times 2.

00:09:33.660 --> 00:09:39.790
So minus 21 plus 2
is minus is 19.

00:09:39.790 --> 00:09:46.020
So the coordinates of a with
respect to the basis B are

00:09:46.020 --> 00:09:48.850
going to be equal to-- Let me
just multiply the negative

00:09:48.850 --> 00:09:52.610
1/5-- you get minus 3/5.

00:09:52.610 --> 00:09:55.230
And then you get plus 19/5.

00:09:55.230 --> 00:10:00.050
So 19 over 5.

00:10:00.050 --> 00:10:00.830
Just like that.

00:10:00.830 --> 00:10:03.480
And let's verify that.

00:10:03.480 --> 00:10:06.490
This means that a is equal to
minus 3/5 times our first

00:10:06.490 --> 00:10:10.580
basis vector plus 19/5 times
our second basis vector.

00:10:10.580 --> 00:10:12.730
Let's verify that
that's the case.

00:10:12.730 --> 00:10:25.330
So let's see, minus 3/5 times
1, 3, plus 19/5 times 2, 1.

00:10:25.330 --> 00:10:27.290
Let's see what this is
going to be equal to.

00:10:31.160 --> 00:10:33.100
Let me write the two vectors.

00:10:33.100 --> 00:10:39.940
This is minus 3/5 times
3 is minus 9/5.

00:10:42.670 --> 00:10:45.210
And there we're going to
add it to this guy.

00:10:45.210 --> 00:10:51.690
So this guy is 2 times
19 is 38/5.

00:10:51.690 --> 00:10:52.340
Right?

00:10:52.340 --> 00:10:56.390
And then 19/5 times 1 is 19/5.

00:10:56.390 --> 00:10:58.850
And then if you add
these two vectors

00:10:58.850 --> 00:11:00.410
together, what do we get?

00:11:00.410 --> 00:11:04.960
We get minus 3/5 plus 38/5.

00:11:04.960 --> 00:11:06.810
That's 35/3.

00:11:06.810 --> 00:11:13.480
35/5 is 7, minus
9/5 plus 19/5.

00:11:13.480 --> 00:11:16.320
That's 10/5 or 2.

00:11:16.320 --> 00:11:17.570
And there you have it.

00:11:17.570 --> 00:11:19.400
That was our original a.

00:11:19.400 --> 00:11:21.650
So we see that a can definitely
be represented as

00:11:21.650 --> 00:11:26.510
minus 3/5 times our first basis
vector plus 19/5 times

00:11:26.510 --> 00:11:28.090
our second basis vector.

00:11:28.090 --> 00:11:31.800
Now that was a case where we
had some vector a and we

00:11:31.800 --> 00:11:34.870
wanted to represent it in
coordinates with respect to B.

00:11:34.870 --> 00:11:36.800
What if we had the other way?

00:11:36.800 --> 00:11:43.180
What if we said that some vector
w's coordinates with

00:11:43.180 --> 00:11:50.170
respect to B are-- I'll do
something simple-- are 1, 1.

00:11:50.170 --> 00:11:54.180
Then what is w in standard
coordinates?

00:11:54.180 --> 00:11:58.120
Well there we can
just multiply.

00:11:58.120 --> 00:12:01.980
Remember w is just equal to
the change of basis matrix

00:12:01.980 --> 00:12:05.640
times w's coordinates with
respect to the basis B.

00:12:05.640 --> 00:12:10.640
So w is going to be equal to
the change of basis matrix,

00:12:10.640 --> 00:12:19.500
which is just 1, 3, 2, 1, times
the coordinates of w

00:12:19.500 --> 00:12:22.930
with respect to B times 1, 1.

00:12:22.930 --> 00:12:27.450
Which is equal to 1 times
1 plus 2 times 1 is 3.

00:12:27.450 --> 00:12:31.270
And then 3 times 1
plus 1 plus 1.

00:12:31.270 --> 00:12:35.390
So 3 times 1 is 3,
plus 1 is 4.

00:12:35.390 --> 00:12:38.380
So w is just equal to
the vector 3, 4.

00:12:38.380 --> 00:12:41.240
So there you see if our change
of basis matrix is invertible,

00:12:41.240 --> 00:12:44.460
which is really just another way
of saying that are basis

00:12:44.460 --> 00:12:49.430
spans Rn-- in this example it
was R2-- then you can easily

00:12:49.430 --> 00:12:52.360
go back and forth between
coordinate representations in

00:12:52.360 --> 00:12:59.560
our standard coordinates and
coordinate representations

00:12:59.560 --> 00:13:02.300
with respect to our basis.

00:13:02.300 --> 00:13:02.620
Right?

00:13:02.620 --> 00:13:03.960
This is with respect
to the basis.

00:13:03.960 --> 00:13:05.210
This is in standard
coordinates.

00:13:08.970 --> 00:13:12.600
And you can do that just simply
by either using this

00:13:12.600 --> 00:13:16.600
information or just saying,
oh, the coordinates with

00:13:16.600 --> 00:13:18.890
respect to the basis equal to
C inverse times a, or the

00:13:18.890 --> 00:13:21.880
inverse of our change of
basis matrix times a.

00:13:21.880 --> 00:13:27.540
Or saying our coordinates with
respect to the standard basis

00:13:27.540 --> 00:13:30.120
is just equal to the change
of basis matrix times the

00:13:30.120 --> 00:13:32.740
coordinates with respect
to the basis.


Change of basis matrix

WEBVTT
Kind: captions
Language: en

00:00:00.710 --> 00:00:04.370
Let's say I've got some
basis B, and it's

00:00:04.370 --> 00:00:05.790
made up of k vectors.

00:00:05.790 --> 00:00:12.050
Let's say it's v1, v2,
all the way to vk.

00:00:12.050 --> 00:00:16.830
Let's say I have some vector
a, and I know what a's

00:00:16.830 --> 00:00:20.600
coordinates are with
respect to B.

00:00:20.600 --> 00:00:26.490
So this is the coordinates of
a with respect to B are c1,

00:00:26.490 --> 00:00:30.800
c2, and I'm going to have k
coordinates, because we have k

00:00:30.800 --> 00:00:31.780
basis vectors.

00:00:31.780 --> 00:00:34.645
Or if this describes a
subspace, this is a

00:00:34.645 --> 00:00:36.670
k-dimensional subspace.

00:00:36.670 --> 00:00:40.790
So I'm going to have k of
these guys right there.

00:00:40.790 --> 00:00:43.740
All this means, by our
definition of coordinates with

00:00:43.740 --> 00:00:47.970
respect to a basis, this
literally means that I can

00:00:47.970 --> 00:00:54.210
represent my vector a as a
linear combination of these

00:00:54.210 --> 00:00:58.120
guys, where these coordinates
are the weights.

00:00:58.120 --> 00:01:04.709
So a would be equal to c1 times
v1, plus c2 times v2,

00:01:04.709 --> 00:01:07.030
plus all the way, keep adding
them up, all the

00:01:07.030 --> 00:01:11.250
way to ck times vk.

00:01:11.250 --> 00:01:17.080
Now, another way to write
this is that-- let me

00:01:17.080 --> 00:01:18.160
write it this way.

00:01:18.160 --> 00:01:22.590
If I had a matrix where the
column vectors were the basis

00:01:22.590 --> 00:01:25.660
vectors of B-- so let me write
it just like that.

00:01:25.660 --> 00:01:30.190
So let me see I have some matrix
C that looks like this,

00:01:30.190 --> 00:01:32.820
where its column vectors are
just these basis vectors.

00:01:32.820 --> 00:01:39.230
So we have v1, v2, all
the way to vk.

00:01:39.230 --> 00:01:42.990
If we assume that all of these
are a member of Rn, then each

00:01:42.990 --> 00:01:47.870
of these are going to have n
entries, or it's going to an n

00:01:47.870 --> 00:01:50.130
by k matrix.

00:01:50.130 --> 00:01:51.670
Each of these guys
have n entries.

00:01:51.670 --> 00:01:54.800
So we're going to have n rows,
and we have k columns.

00:01:54.800 --> 00:01:59.220
So let's imagine this
matrix right there.

00:01:59.220 --> 00:02:02.450
Another way to write this
expression right there, is to

00:02:02.450 --> 00:02:11.670
say that a is equal to the
vector c1, c2, all the way to

00:02:11.670 --> 00:02:15.520
ck, multiplied by this
matrix right there.

00:02:15.520 --> 00:02:17.210
This would be equal to a.

00:02:20.070 --> 00:02:24.650
This statement over here and
this expression over here are

00:02:24.650 --> 00:02:25.620
completely identical.

00:02:25.620 --> 00:02:29.110
If I take this matrix vector
product, what do I get?

00:02:29.110 --> 00:02:36.250
I get c1 times v1, plus c2 times
v2, plus c3 times v3,

00:02:36.250 --> 00:02:38.960
all the way to ck times
vk, is equal to a.

00:02:38.960 --> 00:02:40.950
We've seen this multiple
times in

00:02:40.950 --> 00:02:42.860
multiple different contexts.

00:02:42.860 --> 00:02:44.710
But what's interesting
here is this

00:02:44.710 --> 00:02:46.390
expression is the same thing.

00:02:46.390 --> 00:02:48.710
And really I'm just applying new
words to things that we've

00:02:48.710 --> 00:02:51.350
seen probably 100
times by now.

00:02:51.350 --> 00:02:52.750
We can rewrite this
expression.

00:02:52.750 --> 00:02:57.530
This is C-- and remember C is
just a matrix with our basis

00:02:57.530 --> 00:03:01.400
vectors as columns-- C
is equal to this guy.

00:03:01.400 --> 00:03:04.210
This is just the coordinates
of a with respect

00:03:04.210 --> 00:03:05.750
to the basis B.

00:03:05.750 --> 00:03:11.320
So C times the vector that has
the coordinates of the vector

00:03:11.320 --> 00:03:15.330
a with respect to the basis B.

00:03:15.330 --> 00:03:17.245
That is going to
be equal to a.

00:03:21.000 --> 00:03:23.510
Now, why did I go through the
trouble of doing this?

00:03:23.510 --> 00:03:26.820
Because now you have a fairly
straightforward way of-- if I

00:03:26.820 --> 00:03:29.760
were to give you this, if I were
to give you that right

00:03:29.760 --> 00:03:36.600
there, and say, hey, what is a,
if I wanted to write it in

00:03:36.600 --> 00:03:39.710
standard coordinates, or with
respect to the standard basis?

00:03:39.710 --> 00:03:41.120
Which is just kind of
the way we've been

00:03:41.120 --> 00:03:42.560
writing vectors all along?

00:03:42.560 --> 00:03:45.140
Then you just multiply it times
this matrix C, this

00:03:45.140 --> 00:03:48.750
matrix that has the basis
vectors as columns.

00:03:48.750 --> 00:03:55.490
The other way, if you have some
vector a that you know

00:03:55.490 --> 00:03:59.010
can be represented as a linear
combination of B, or it's in

00:03:59.010 --> 00:04:02.920
the span of these basis vectors,
then you could solve

00:04:02.920 --> 00:04:06.540
for this guy right here to
figure out a's coordinates

00:04:06.540 --> 00:04:08.520
with respect to B.

00:04:08.520 --> 00:04:11.200
So this little matrix right
here, what does it do?

00:04:11.200 --> 00:04:12.730
It helps us change bases.

00:04:12.730 --> 00:04:16.230
If you multiply it times this
guy, you're going from the

00:04:16.230 --> 00:04:19.040
vector represented by
coordinates with respect to

00:04:19.040 --> 00:04:21.100
some basis, and you multiply
it times this guy, you're

00:04:21.100 --> 00:04:24.430
going to get to the vector
just with standard

00:04:24.430 --> 00:04:24.840
coordinates.

00:04:24.840 --> 00:04:35.360
So we call this matrix right
here change of basis matrix,

00:04:35.360 --> 00:04:37.020
which sounds very fancy.

00:04:37.020 --> 00:04:41.090
But all it literally is is a
matrix with the basis vectors

00:04:41.090 --> 00:04:41.970
as columns.

00:04:41.970 --> 00:04:44.230
Let's just apply this a little
bit to see if we can do

00:04:44.230 --> 00:04:47.400
anything vaguely constructive
with it.

00:04:47.400 --> 00:04:51.130
Let's say that I have
some basis.

00:04:51.130 --> 00:04:53.350
let's say B for basis.

00:04:53.350 --> 00:04:55.440
Let's say I have two vectors.

00:04:55.440 --> 00:04:56.930
I'll define the vectors
up here.

00:04:56.930 --> 00:05:00.740
Let's say vector 1, let's say
we're dealing with R3.

00:05:00.740 --> 00:05:04.130
So vector 1 is 1, 2, 3.

00:05:04.130 --> 00:05:12.360
And let's say that vector
2 is 1, 0, 1.

00:05:12.360 --> 00:05:15.540
And let's say I'm going to
define some basis B as being

00:05:15.540 --> 00:05:22.060
the set of the vectors
v1 and v2.

00:05:22.060 --> 00:05:25.800
I'll leave it to you to verify
that these are not linear

00:05:25.800 --> 00:05:29.390
combinations of each other,
so this is a valid basis.

00:05:29.390 --> 00:05:31.740
These aren't in any way
linearly dependent.

00:05:31.740 --> 00:05:35.050
Now, let's say that I know some
vector that's in the span

00:05:35.050 --> 00:05:36.300
of these guys.

00:05:38.790 --> 00:05:41.700
All I know is how it happens
to be represented in

00:05:41.700 --> 00:05:44.710
coordinates with respect
to this basis.

00:05:44.710 --> 00:05:47.620
So let's say I have
some vector a.

00:05:47.620 --> 00:05:51.840
And when I represent the
coordinates of a with respect

00:05:51.840 --> 00:06:02.800
to this basis, it's equal
to 7, 7, minus 4.

00:06:02.800 --> 00:06:06.410
So how can we represent this
guy in its standard

00:06:06.410 --> 00:06:08.390
coordinates?

00:06:08.390 --> 00:06:12.410
What is a equal to?

00:06:12.410 --> 00:06:17.990
Well, you could just say a is
equal to 7 times v1, minus 4

00:06:17.990 --> 00:06:20.720
times v2, and you'd be
completely correct.

00:06:20.720 --> 00:06:23.660
But let's actually use this
change of basis matrix that

00:06:23.660 --> 00:06:25.490
I've introduced you
to in this video.

00:06:25.490 --> 00:06:30.820
So the change of basis matrix
here is going to be just a

00:06:30.820 --> 00:06:35.880
matrix with v1 and v2 as
its columns, 1, 2, 3,

00:06:35.880 --> 00:06:38.550
and then 1, 0, 1.

00:06:38.550 --> 00:06:43.000
And then if we multiply our
change of basis matrix times

00:06:43.000 --> 00:06:47.340
the vector representation with
respect to that basis, so

00:06:47.340 --> 00:06:57.220
times 7 minus 4, we're going to
get the vector represented

00:06:57.220 --> 00:06:59.080
in standard coordinates.

00:06:59.080 --> 00:07:01.650
So what is this going
to be equal to?

00:07:01.650 --> 00:07:05.810
We have a 3 by 2 matrix,
times a 2 by 1.

00:07:05.810 --> 00:07:10.420
We're going to get a 3 by 1
matrix, which makes complete

00:07:10.420 --> 00:07:11.680
sense because we're
dealing in R3.

00:07:15.540 --> 00:07:19.130
a is going to be member of R3.

00:07:19.130 --> 00:07:20.970
So when we write it with
standard coordinates, we

00:07:20.970 --> 00:07:23.680
should have 3 coordinates
right there.

00:07:23.680 --> 00:07:26.520
Now when we represented a with
respect to the basis, we only

00:07:26.520 --> 00:07:30.420
had two coordinates, because a
was in the plane spanned by

00:07:30.420 --> 00:07:31.230
these two guys.

00:07:31.230 --> 00:07:35.460
Actually this is a good
excuse to draw this.

00:07:35.460 --> 00:07:37.270
So let me draw it in
three dimensions.

00:07:37.270 --> 00:07:39.790
Let's say the span of v1
and v2 looks like this.

00:07:45.530 --> 00:07:48.390
Let's say this is the 0
vector right there.

00:07:48.390 --> 00:07:54.620
So this right here is the
span of v1 and v2.

00:07:54.620 --> 00:07:57.290
Or another way, this is
the subspace that B

00:07:57.290 --> 00:07:59.210
is the basis for.

00:07:59.210 --> 00:08:01.490
So we know that a
is in this guy.

00:08:01.490 --> 00:08:07.390
So let's say v1 looks like this,
and that v2-- I'm not

00:08:07.390 --> 00:08:09.810
even looking at the numbers,
I'm just doing it fairly

00:08:09.810 --> 00:08:16.960
abstract-- let's say v2 looks
like this right here.

00:08:16.960 --> 00:08:20.150
Now, the fact that a can be
represented as a linear

00:08:20.150 --> 00:08:23.240
combination of v1 and v2, tells
us that a is also going

00:08:23.240 --> 00:08:26.060
to be in this plane in R3.

00:08:26.060 --> 00:08:32.500
In fact, it's 7 times v1,
so it's 1 v1, 1 v1's, 3

00:08:32.500 --> 00:08:35.549
v1's, 4, 5, 6, 7.

00:08:35.549 --> 00:08:38.390
So it's 7 in that direction, and
then it's minus 4 in the

00:08:38.390 --> 00:08:39.070
v2 direction.

00:08:39.070 --> 00:08:40.635
So that's 1 in the
v2 direction.

00:08:40.635 --> 00:08:43.980
This is minus 1 in the v2
direction, minus 2,

00:08:43.980 --> 00:08:45.850
minus 3, minus 4.

00:08:45.850 --> 00:08:50.470
Or we can do it here,
1, 2, 3, 4.

00:08:50.470 --> 00:08:54.390
So our vector a is going
to look like this.

00:08:54.390 --> 00:08:57.470
It's going to sit
on the plane.

00:08:57.470 --> 00:08:59.680
So this is our vector a.

00:08:59.680 --> 00:09:01.000
It's going to sit
on the plane.

00:09:01.000 --> 00:09:04.190
And when we represent it with
respect to this basis, when we

00:09:04.190 --> 00:09:06.080
represent these coordinates with
respect to our basis B,

00:09:06.080 --> 00:09:08.850
we say oh OK, it's
7 of this guy.

00:09:08.850 --> 00:09:10.060
I'm just doing this
abstractly.

00:09:10.060 --> 00:09:11.570
Don't pay attention to
the numbers just now.

00:09:11.570 --> 00:09:13.480
I just want you to understand
the idea.

00:09:13.480 --> 00:09:17.500
We said it's 7 of this guy,
minus 4 of this guy.

00:09:17.500 --> 00:09:18.590
So it takes you back here.

00:09:18.590 --> 00:09:20.990
And you get this vector,
which is in this plane.

00:09:20.990 --> 00:09:23.310
So we only needed two
coordinates to specify it

00:09:23.310 --> 00:09:26.290
within this plane, because
this subspace was

00:09:26.290 --> 00:09:27.340
two-dimensional.

00:09:27.340 --> 00:09:30.430
But we're dealing in R3.

00:09:30.430 --> 00:09:33.460
And if we just want the general
version of a in

00:09:33.460 --> 00:09:37.030
standard coordinates, we'll have
to essentially get three

00:09:37.030 --> 00:09:38.740
coordinates.

00:09:38.740 --> 00:09:40.730
I want you to understand that
a is sitting on this plane.

00:09:40.730 --> 00:09:43.900
This plane just keeps going on
and on and on in all of these

00:09:43.900 --> 00:09:44.430
directions.

00:09:44.430 --> 00:09:45.840
a actually sits on that plane.

00:09:45.840 --> 00:09:48.070
It's a linear combination of
that guy and that guy.

00:09:48.070 --> 00:09:49.550
But let's figure out what
a looks like in standard

00:09:49.550 --> 00:09:50.690
coordinates.

00:09:50.690 --> 00:09:53.230
In standard coordinates, we get
the first term is going to

00:09:53.230 --> 00:09:56.730
be 1 times 7, plus
1 times minus 4.

00:09:56.730 --> 00:09:58.710
So that's going to be 3.

00:09:58.710 --> 00:10:01.920
We get 2 times 7, plus
0 times minus 4.

00:10:01.920 --> 00:10:03.480
That is 14.

00:10:03.480 --> 00:10:07.610
You're going to get 3 times
7, plus 1 times minus 4.

00:10:07.610 --> 00:10:13.360
So 3 times 7 is 21,
minus 4, is 17.

00:10:13.360 --> 00:10:17.560
So a is the vector 3, 14, 17.

00:10:17.560 --> 00:10:19.430
That is equal to a.

00:10:19.430 --> 00:10:22.780
Now let's say we wanted
to go the other way.

00:10:22.780 --> 00:10:26.400
Let's say we have some vector--
let me pick a letter

00:10:26.400 --> 00:10:29.800
I haven't used recently-- let's
say I have some vector

00:10:29.800 --> 00:10:39.190
d, which is 8, minus 6, 2.

00:10:39.190 --> 00:10:48.220
And let's say we know that d is
a member of the span of our

00:10:48.220 --> 00:10:56.600
basis vectors, the span of v1
and v2, which tells us that d

00:10:56.600 --> 00:10:59.380
can be represented as a linear
combination of these guys, or

00:10:59.380 --> 00:11:02.990
that d is in this subspace, or
that d can be represented as

00:11:02.990 --> 00:11:06.040
coordinates with respect
to the basis B.

00:11:06.040 --> 00:11:13.680
Remember, the basis B was just
equal to the set of v1 and v2.

00:11:13.680 --> 00:11:15.450
That's all that basis B was.

00:11:15.450 --> 00:11:22.410
Now, we know that if we have
our change of basis matrix

00:11:22.410 --> 00:11:28.610
times the vector made up of
the coordinates, of d with

00:11:28.610 --> 00:11:33.660
respect to B-- so let me write
that down, d with respect to

00:11:33.660 --> 00:11:37.080
B-- is equal to d.

00:11:37.080 --> 00:11:37.680
We know that.

00:11:37.680 --> 00:11:40.285
We know if we have this guy's
coordinates and we multiply it

00:11:40.285 --> 00:11:44.270
by the change of basis matrix,
we'll just get the regular

00:11:44.270 --> 00:11:46.900
standard coordinate
representation of d.

00:11:46.900 --> 00:11:49.370
Now in this case, we have d.

00:11:49.370 --> 00:11:50.690
We're given this.

00:11:50.690 --> 00:11:53.450
We of course know what the
change of basis matrix is.

00:11:53.450 --> 00:11:57.640
So if we wanted to represent d
in coordinates with respect to

00:11:57.640 --> 00:11:59.940
B, we're going to have to
solve this equation.

00:11:59.940 --> 00:12:01.200
So let's do that.

00:12:01.200 --> 00:12:08.050
So our change of basis matrix
is 1, 1, 2, 0, 3, 1.

00:12:08.050 --> 00:12:10.580
And we're going to have to
multiply it times some

00:12:10.580 --> 00:12:11.830
coordinates.

00:12:11.830 --> 00:12:14.640
This thing right here, we can
represent it as-- I'll do it

00:12:14.640 --> 00:12:18.050
in yellow-- we're going to
need two coordinates.

00:12:18.050 --> 00:12:21.180
It's going to be some
multiple of v1, plus

00:12:21.180 --> 00:12:22.450
some multiple of v2.

00:12:22.450 --> 00:12:24.940
So it's c1, c2.

00:12:24.940 --> 00:12:26.870
We know it has to be two
coordinates because this

00:12:26.870 --> 00:12:30.210
matrix vector product is only
well-defined if this is a

00:12:30.210 --> 00:12:33.920
member of R2, because this
is a 3 by 2 matrix.

00:12:33.920 --> 00:12:35.190
We have two columns here,
so we have to

00:12:35.190 --> 00:12:36.890
have two entries here.

00:12:36.890 --> 00:12:38.895
Then that's going to
be equal to d.

00:12:41.955 --> 00:12:45.530
So we have 8, minus 6, 2.

00:12:45.530 --> 00:12:47.940
So if we figure out what this
vector is, we've figured out

00:12:47.940 --> 00:12:51.060
what the representation, or
the coordinates of d with

00:12:51.060 --> 00:12:52.050
respect to B, are.

00:12:52.050 --> 00:12:53.300
So let's solve this.

00:12:56.030 --> 00:13:01.150
So to solve this, we can just
set up an augmented matrix.

00:13:01.150 --> 00:13:02.940
That's just our traditional
way of

00:13:02.940 --> 00:13:04.610
solving a linear equation.

00:13:04.610 --> 00:13:11.340
So we have 1, 1, 2, 0, 3, 1.

00:13:11.340 --> 00:13:13.740
We augment it with this
side right there.

00:13:13.740 --> 00:13:17.850
So we have 8, minus 6, and 2.

00:13:17.850 --> 00:13:20.490
And let's keep my first
row the same.

00:13:20.490 --> 00:13:24.560
So I have 1, 1, augmented
it with 8.

00:13:24.560 --> 00:13:30.940
And let's replace my second row
with the second row minus

00:13:30.940 --> 00:13:33.070
2 times the first row.

00:13:33.070 --> 00:13:36.770
So I'm going to get 2 minus 2
times 1 is-- actually, let me

00:13:36.770 --> 00:13:37.400
do it the other way.

00:13:37.400 --> 00:13:40.360
Let me replace my second row
with 2 times my first row,

00:13:40.360 --> 00:13:42.130
minus my second row.

00:13:42.130 --> 00:13:45.990
So two times 1, minus 2, is 0.

00:13:45.990 --> 00:13:50.650
2 times 1, minus 0, is 2.

00:13:50.650 --> 00:13:55.390
2 times 8 is 16,
minus 6, is 10.

00:13:55.390 --> 00:13:59.960
Now let's replace the third
row with 3 times the first

00:13:59.960 --> 00:14:01.820
row, minus the third row.

00:14:01.820 --> 00:14:05.190
So 3 times 1, minus 3, is 0.

00:14:05.190 --> 00:14:09.660
3 times 1, minus 1, is 2.

00:14:09.660 --> 00:14:23.290
And then 3 times 8 is 24, minus
2, is going to be 22.

00:14:23.290 --> 00:14:25.540
See it looks like I must have
made a mistake someplace,

00:14:25.540 --> 00:14:28.180
because I have these two would
lead to no solutions.

00:14:28.180 --> 00:14:30.640
Let me verify what I did, make
sure that I didn't make any

00:14:30.640 --> 00:14:31.990
strange errors.

00:14:31.990 --> 00:14:34.950
So the second row, I replaced it
with 2 times the first row,

00:14:34.950 --> 00:14:36.110
minus the second row.

00:14:36.110 --> 00:14:39.740
So 2 times 1, minus 2, is 0.

00:14:39.740 --> 00:14:41.920
2 times 1, minus 0, is 2.

00:14:41.920 --> 00:14:46.620
2 times 8, minus minus 6-- so
there's my error-- that's

00:14:46.620 --> 00:14:49.230
equal to 22.

00:14:49.230 --> 00:14:49.790
That was my error.

00:14:49.790 --> 00:14:51.080
So these two things
are equivalent.

00:14:56.000 --> 00:14:57.305
I'll do one step at a time.

00:15:00.650 --> 00:15:03.910
Let me replace my third row with
my third row, minus my

00:15:03.910 --> 00:15:05.260
second row, just get
it out of the way.

00:15:05.260 --> 00:15:11.130
So I"ll keep this 1,
1, 8, 0, 2, 22.

00:15:11.130 --> 00:15:14.570
And then the third row, I'm
going to replace it with my

00:15:14.570 --> 00:15:16.110
third row, minus
my second row.

00:15:16.110 --> 00:15:18.130
So it's going to be 0, 0, 0.

00:15:18.130 --> 00:15:20.200
So that just gets zeroed out.

00:15:20.200 --> 00:15:25.260
Now let me divide my
second row by 2.

00:15:25.260 --> 00:15:28.060
So I get 1, 1, and 8.

00:15:28.060 --> 00:15:31.250
And then this one becomes
0, 1, and 11.

00:15:31.250 --> 00:15:34.470
Then of course the third row
is just a bunch of 0's.

00:15:34.470 --> 00:15:38.300
Then let me keep my middle
row the same.

00:15:38.300 --> 00:15:42.290
So it's 0, 1, and 11.

00:15:42.290 --> 00:15:45.790
Then let me replace my first
with my first row, minus my

00:15:45.790 --> 00:15:46.770
middle row.

00:15:46.770 --> 00:15:48.950
So 1 minus 0 is 1.

00:15:48.950 --> 00:15:51.070
1 minus 1 is 0.

00:15:51.070 --> 00:15:53.600
8 minus 11 is minus 3.

00:15:53.600 --> 00:15:55.540
And I'll keep my last
row the same.

00:15:55.540 --> 00:15:58.550
So I put the left-hand side in
reduced row echelon form.

00:15:58.550 --> 00:15:59.960
So this right here
is essentially

00:15:59.960 --> 00:16:01.210
telling me my solution.

00:16:04.350 --> 00:16:05.830
So I could write it this way.

00:16:05.830 --> 00:16:18.480
I could write that 1, 0, 0, 1,
0, 0 times c1, c2 is equal to

00:16:18.480 --> 00:16:21.030
minus 3, 11, 0.

00:16:21.030 --> 00:16:25.420
Or another way of writing this,
is that 1 times c1, plus

00:16:25.420 --> 00:16:30.060
0 times c2, or c1, is
equal to minus 3.

00:16:35.540 --> 00:16:42.700
Then we have 0 times c1, plus
1 times c2 is going to be

00:16:42.700 --> 00:16:45.510
equal to 11.

00:16:45.510 --> 00:16:53.350
So our solution to this equation
is minus 3, 11.

00:16:53.350 --> 00:16:56.900
Or another way of saying this,
is that if I wanted to write

00:16:56.900 --> 00:17:01.470
my vector d in coordinates with
respect to my basis B, it

00:17:01.470 --> 00:17:19.319
would be the coordinates minus
3, 11, which implies-- let me

00:17:19.319 --> 00:17:23.060
write it this way-- which
implies that d is equal to

00:17:23.060 --> 00:17:27.670
minus 3 times vector 1, plus
11, times vector 2.

00:17:27.670 --> 00:17:30.520
I'll leave that for
you to verify.

00:17:30.520 --> 00:17:34.190
But just like that, using this
change of basis matrix, we can

00:17:34.190 --> 00:17:35.130
go back and forth.

00:17:35.130 --> 00:17:37.270
If you have this representation,
it's very easy

00:17:37.270 --> 00:17:39.540
to take the product and
get the standard

00:17:39.540 --> 00:17:40.990
representation for d.

00:17:40.990 --> 00:17:43.700
If you have the standard
representation or the

00:17:43.700 --> 00:17:45.780
coordinates with respect
to the standard

00:17:45.780 --> 00:17:47.310
basis, it's very easy.

00:17:47.310 --> 00:17:48.830
Well, it's a little
more involved.

00:17:48.830 --> 00:17:50.870
But then you just solve
for your coordinates

00:17:50.870 --> 00:17:53.930
with respect to B.


Coordinates with respect to a basis

WEBVTT
Kind: captions
Language: en

00:00:00.720 --> 00:00:03.680
Let's say I've got some
subspace of Rn.

00:00:03.680 --> 00:00:11.790
Let's say V is a
subspace of Rn.

00:00:11.790 --> 00:00:16.350
And let's say the set B-- I'll
do it in blue-- let's say the

00:00:16.350 --> 00:00:20.880
set B is a basis for V,
so it's got a bunch

00:00:20.880 --> 00:00:21.930
of vectors in it.

00:00:21.930 --> 00:00:28.480
And let's say it's got v1,
v2, all the way to vk.

00:00:28.480 --> 00:00:31.660
So you can see we have
k vectors, so v is a

00:00:31.660 --> 00:00:35.790
k-dimensional subspace.

00:00:35.790 --> 00:00:40.960
So that means that if I have
some vector a-- let's say I

00:00:40.960 --> 00:00:44.910
have some vector a that is a
member of my subspace-- that

00:00:44.910 --> 00:00:47.640
means that I can represent a
as a linear combination of

00:00:47.640 --> 00:00:49.840
these characters right there.

00:00:49.840 --> 00:00:54.130
So I could write a as being
equal to some constant times

00:00:54.130 --> 00:00:58.420
my first basis vector, plus some
other constant, times my

00:00:58.420 --> 00:01:00.140
second basis vector.

00:01:00.140 --> 00:01:03.310
And then I can keep going all
the way to a kth constant

00:01:03.310 --> 00:01:05.940
times my k basis vector.

00:01:05.940 --> 00:01:09.640
Now, I've used the term
coordinates fairly loosely in

00:01:09.640 --> 00:01:12.060
the past. And now we're going
to have a more precise

00:01:12.060 --> 00:01:13.010
definition.

00:01:13.010 --> 00:01:25.470
I am going to call these
constants here c1, c2, all the

00:01:25.470 --> 00:01:27.620
way through ck.

00:01:27.620 --> 00:01:31.500
I'm going to call them the--
I'll do it in a new color--

00:01:31.500 --> 00:01:47.750
the coordinates of a, with
respect to our basis B, with

00:01:47.750 --> 00:01:49.170
respect to B.

00:01:49.170 --> 00:01:50.930
And we could also write
it like this.

00:01:50.930 --> 00:01:55.930
We could also write I have
my vector a, so I

00:01:55.930 --> 00:01:57.740
have my vector a.

00:01:57.740 --> 00:02:01.610
But if I wanted to write my
vector a in coordinates with

00:02:01.610 --> 00:02:05.800
respect to this basis set B,
I would write it like this.

00:02:05.800 --> 00:02:09.690
Put brackets around it, and put
the basis set right there.

00:02:09.690 --> 00:02:12.200
And this says I'm going to write
this in coordinates with

00:02:12.200 --> 00:02:14.340
respect to this basis set.

00:02:14.340 --> 00:02:18.570
So, I will then write
it like this.

00:02:18.570 --> 00:02:23.150
I'm just going to put these
weights there, these constant

00:02:23.150 --> 00:02:26.790
terms, on the on the linear
combination that I have to get

00:02:26.790 --> 00:02:28.910
of my basis vectors to get a.

00:02:28.910 --> 00:02:34.880
So there's c1, c2, all
the way to ck.

00:02:34.880 --> 00:02:37.600
Now there's one slightly
interesting thing, or maybe

00:02:37.600 --> 00:02:40.680
very interesting thing,
to point out here.

00:02:40.680 --> 00:02:43.520
V is as basis of Rn, so
anything in V is also

00:02:43.520 --> 00:02:45.620
going to be in Rn.

00:02:45.620 --> 00:02:47.360
But V has k vectors.

00:02:47.360 --> 00:02:48.390
It has dimension k.

00:02:48.390 --> 00:02:51.090
And that k could be as high
as n, but it might

00:02:51.090 --> 00:02:53.550
be something smaller.

00:02:53.550 --> 00:02:57.410
Maybe we have two vectors in R3,
in which case v would be a

00:02:57.410 --> 00:02:59.920
plane in R3, but we can abstract
that to further

00:02:59.920 --> 00:03:00.740
dimensions.

00:03:00.740 --> 00:03:05.050
But when you specify something
that is in our subspace, with

00:03:05.050 --> 00:03:09.610
respect to its basis, notice
you only have to have that

00:03:09.610 --> 00:03:13.840
many-- the dimension of your
subspaces-- you only have to

00:03:13.840 --> 00:03:15.830
have that many coordinates.

00:03:15.830 --> 00:03:19.090
So even though a is a member of
Rn, I only have to give it

00:03:19.090 --> 00:03:19.920
k coordinates.

00:03:19.920 --> 00:03:23.720
Because, essentially, you're
giving it positions within,

00:03:23.720 --> 00:03:25.850
let's say, if this was a plane,
within the plane that

00:03:25.850 --> 00:03:27.850
is your subspace.

00:03:27.850 --> 00:03:30.780
Let me make this a little
bit more concrete.

00:03:30.780 --> 00:03:32.440
Let me do some examples.

00:03:32.440 --> 00:03:34.960
So let's say we have
some subspace.

00:03:34.960 --> 00:03:37.330
Let me clear this out.

00:03:37.330 --> 00:03:39.280
Let's say I have a couple
of vectors.

00:03:39.280 --> 00:03:44.170
Let's say v1 is the
vector 2, 1.

00:03:44.170 --> 00:03:52.710
And let's say v2 is
the vector 1, 2.

00:03:52.710 --> 00:03:58.740
Now you might immediately see
that the basis, or the set, of

00:03:58.740 --> 00:04:10.230
v1 and v2, this is a basis for
R2, which means that any

00:04:10.230 --> 00:04:12.360
vector in R2 can be represented
as a linear

00:04:12.360 --> 00:04:13.540
combination of these guys.

00:04:13.540 --> 00:04:15.240
I could do a visual argument.

00:04:15.240 --> 00:04:19.360
Or we also know that, look, R2
is two-dimensional, and we

00:04:19.360 --> 00:04:22.290
have two basic vectors right
here, and they are linearly

00:04:22.290 --> 00:04:22.860
independent.

00:04:22.860 --> 00:04:23.880
You can verify that.

00:04:23.880 --> 00:04:26.120
In fact, the easiest way to
verify that, is if you just

00:04:26.120 --> 00:04:30.450
take 2, 1 and 1, 2, and you put
it in reduced row echelon

00:04:30.450 --> 00:04:34.620
form, you're going to get the
2 by 2 identity matrix.

00:04:34.620 --> 00:04:39.260
You're going to get 1, 0, 0, 1,
and that lets you know that

00:04:39.260 --> 00:04:42.770
this guy and this guy are
both basis vectors.

00:04:42.770 --> 00:04:44.200
So, that's all review.

00:04:44.200 --> 00:04:45.460
We've seen that before.

00:04:45.460 --> 00:04:47.320
But let's visualize these.

00:04:47.320 --> 00:04:49.530
Let's visualize these guys.

00:04:49.530 --> 00:04:52.450
So if I were to just graph it
in the way we normally graph

00:04:52.450 --> 00:04:54.660
these vectors, what does
2, 1 look like? let me

00:04:54.660 --> 00:04:56.960
draw some axes here.

00:04:56.960 --> 00:04:57.970
Let me draw it.

00:04:57.970 --> 00:04:59.220
We'll do it in a different
color.

00:05:02.010 --> 00:05:06.640
Let's say that is my vertical
axis, and this is my

00:05:06.640 --> 00:05:09.820
horizontal axis.

00:05:09.820 --> 00:05:12.520
And 2, 1 might look like this.

00:05:12.520 --> 00:05:15.710
We're going to go out
1, 2, and then we're

00:05:15.710 --> 00:05:17.270
going to go up 1.

00:05:17.270 --> 00:05:20.460
So that is our vector
1, right there.

00:05:20.460 --> 00:05:23.160
That is 2, 1, that's
our vector 1.

00:05:23.160 --> 00:05:27.670
And then 1, 2 might look like,
or it does look like this, if

00:05:27.670 --> 00:05:28.680
I draw it in a standard
position.

00:05:28.680 --> 00:05:30.680
1, then you go up 2.

00:05:30.680 --> 00:05:31.980
1, 2 looks like this.

00:05:35.740 --> 00:05:38.200
So when we talk about
coordinates with respect to

00:05:38.200 --> 00:05:44.290
this basis, let me pick
some member of R2.

00:05:44.290 --> 00:05:47.510
I'll engineer it so that I
can easily find a linear

00:05:47.510 --> 00:05:48.150
combination.

00:05:48.150 --> 00:06:00.210
Let me take 3 times v1,
plus 2 times v2.

00:06:00.210 --> 00:06:01.960
What is that going
to be equal to?

00:06:01.960 --> 00:06:04.710
That's going to be equal
to the vector.

00:06:04.710 --> 00:06:09.950
So 3 times 2, which is
6, plus 2 times 1.

00:06:09.950 --> 00:06:15.630
So it's the vector 8, and then
I have the vector 3 times v1,

00:06:15.630 --> 00:06:16.900
plus 2 times that.

00:06:16.900 --> 00:06:18.300
So 8, 7, right?

00:06:18.300 --> 00:06:23.230
3 plus 2 times 2 is for 8, 7.

00:06:23.230 --> 00:06:26.215
So if we were to just graph 8,
7 in the traditional way, we

00:06:26.215 --> 00:06:33.530
would go 1, 2, 3, 4, 5, 6, 7, 8,
and then we would go up 1,

00:06:33.530 --> 00:06:37.870
2, 3, 4, 5, 6, 7.

00:06:37.870 --> 00:06:40.190
And we would have a vector, I'm
not going to draw it out

00:06:40.190 --> 00:06:45.740
here, but it would specify
that point right there.

00:06:45.740 --> 00:06:47.080
That would be this point.

00:06:47.080 --> 00:06:49.590
If you view these as
coordinates, we would view

00:06:49.590 --> 00:06:51.940
that as point 8,
7 right there.

00:06:51.940 --> 00:06:53.100
Maybe I'll write it like that.

00:06:53.100 --> 00:06:54.240
That's the point 8, 7.

00:06:54.240 --> 00:06:56.050
If I wanted to draw the spectrum
in standard position,

00:06:56.050 --> 00:06:58.820
I would draw a vector that
ends right there.

00:06:58.820 --> 00:07:02.170
Now, we have this is basis
here, this basis B,

00:07:02.170 --> 00:07:04.190
represented by these
two vectors.

00:07:04.190 --> 00:07:05.635
This is B1 and B2.

00:07:05.635 --> 00:07:08.980
And what we want to do is
represent this guy.

00:07:08.980 --> 00:07:13.640
Let's say that this is vector--
let me call that

00:07:13.640 --> 00:07:19.210
vector a-- so vector
a is equal to 8, 7.

00:07:19.210 --> 00:07:23.350
Now, we know that if we wanted
to represent vector a as a

00:07:23.350 --> 00:07:26.070
linear combination of my basis
vectors, it's going to be 3

00:07:26.070 --> 00:07:28.910
times you v1, plus 2 times v2.

00:07:28.910 --> 00:07:31.610
So just given what we just saw
in the earlier part of this

00:07:31.610 --> 00:07:38.190
video, we can write that the
vector a, with respect to the

00:07:38.190 --> 00:07:41.660
basis B-- maybe I'll do it in
the same color as the basis--

00:07:41.660 --> 00:07:46.960
with respect to the basis B, is
equal to these weights on

00:07:46.960 --> 00:07:51.655
the basis vectors, is
equal to 3 and 2.

00:07:51.655 --> 00:07:54.700
And let's see if we can visually
understand why this

00:07:54.700 --> 00:07:55.770
makes sense.

00:07:55.770 --> 00:07:59.930
We're saying that in some new
coordinate system, where this

00:07:59.930 --> 00:08:02.390
vector can be represented as 3,
2-- and the way you think

00:08:02.390 --> 00:08:05.000
about the new coordinate
system is, in this old

00:08:05.000 --> 00:08:07.960
coordinate system we hashed
out 1's in the horizontal

00:08:07.960 --> 00:08:10.350
axis, and that was our
first coordinate.

00:08:10.350 --> 00:08:13.130
And we hashed out 1's in
the vertical direction.

00:08:13.130 --> 00:08:14.590
That was our second
coordinate.

00:08:14.590 --> 00:08:18.480
Now in our new system, what's
our first coordinate?

00:08:18.480 --> 00:08:23.810
Our first coordinate is going
to be multiples of v1.

00:08:23.810 --> 00:08:26.540
This is v1 or this is v1,
so it's multiples of v1.

00:08:26.540 --> 00:08:28.540
So that's 1 times v1.

00:08:28.540 --> 00:08:31.680
Then if we do 2 times v1, we're
going to get, over here,

00:08:31.680 --> 00:08:34.049
2 times v1 would get
us to, what?

00:08:34.049 --> 00:08:38.400
2 would get us to 4, 2.

00:08:38.400 --> 00:08:42.289
3 times v1 would
get us to 6, 3.

00:08:42.289 --> 00:08:48.150
So let's see, 1, 2,
3, 4, 5, 6, 7, 8.

00:08:48.150 --> 00:08:52.780
So it's 6 and then 3,
just like that.

00:08:52.780 --> 00:08:57.945
And then 4 times v1 would
get us to 8 and 4.

00:08:57.945 --> 00:09:01.330
So you can imagine that what I'm
drawing here, this is kind

00:09:01.330 --> 00:09:05.670
of the axes, the first-term
axis generated by v1.

00:09:05.670 --> 00:09:09.450
So I could draw it-- let me do
it in this blue color-- so you

00:09:09.450 --> 00:09:12.946
can imagine it like this.

00:09:12.946 --> 00:09:16.990
This would be a straight
line, just like that.

00:09:16.990 --> 00:09:20.090
And then the coordinate tells me
how many v1's do I have. So

00:09:20.090 --> 00:09:22.830
I would hash off the coordinate
system like this.

00:09:22.830 --> 00:09:24.490
Instead of doing increments
of 1, I'm going to do

00:09:24.490 --> 00:09:27.700
increments of v1.

00:09:27.700 --> 00:09:28.780
I'll write it just like that.

00:09:28.780 --> 00:09:31.360
As you go 9, 10, we're going
to go up one more to 5,

00:09:31.360 --> 00:09:33.800
something like that.

00:09:33.800 --> 00:09:37.870
Now, the second coordinate tells
you increments of v2.

00:09:37.870 --> 00:09:41.080
So this is our first increment
of v2, then our second

00:09:41.080 --> 00:09:44.080
increment if we go to 4,
is going to be 4, 2.

00:09:44.080 --> 00:09:44.850
Just like that.

00:09:44.850 --> 00:09:47.892
That's going to be 6 and 3.

00:09:47.892 --> 00:09:50.060
It's going to be
just like that.

00:09:50.060 --> 00:09:52.560
It's going to be 6 and 3.

00:09:52.560 --> 00:09:55.970
So it's going to look
something like this.

00:09:55.970 --> 00:09:58.830
So if you want to think of it
is a bit of a, well, you

00:09:58.830 --> 00:10:00.620
should be thinking of it
as a coordinate system.

00:10:00.620 --> 00:10:05.570
You can have this new skewed
graph paper, where any point

00:10:05.570 --> 00:10:09.320
you can now specify it as going
in the v1 direction by

00:10:09.320 --> 00:10:11.350
some amount, and then
by going in the v2

00:10:11.350 --> 00:10:12.280
direction by some amount.

00:10:12.280 --> 00:10:14.130
Let me draw that as
this graph paper.

00:10:14.130 --> 00:10:19.980
So I could draw another version
of v2, just like that,

00:10:19.980 --> 00:10:22.500
just all the multiples of v2.

00:10:22.500 --> 00:10:24.100
I could shift them like that.

00:10:24.100 --> 00:10:26.760
I could do another
one like this.

00:10:26.760 --> 00:10:28.140
I could do another.

00:10:28.140 --> 00:10:31.035
That one is a little bit
not neat enough.

00:10:31.035 --> 00:10:33.630
I could do it like that.

00:10:33.630 --> 00:10:37.280
I could do-- I think you're
getting the idea.

00:10:37.280 --> 00:10:39.630
Let me make that a little
bit neater.

00:10:39.630 --> 00:10:42.430
This might have been useful
to do with another tool.

00:10:42.430 --> 00:10:45.500
And then I could do all the
multiples of v1 like this.

00:10:45.500 --> 00:10:47.320
I'm doing a graph paper
right here.

00:10:47.320 --> 00:10:50.350
So it looks something
like this.

00:10:50.350 --> 00:10:52.980
It would look something
like this.

00:10:52.980 --> 00:10:55.560
It would look something
like this.

00:10:55.560 --> 00:10:58.090
And so you can imagine the
skewed graph paper, if I did

00:10:58.090 --> 00:10:59.740
it all over the place
with this kind of

00:10:59.740 --> 00:11:01.160
green and this blue.

00:11:01.160 --> 00:11:03.390
So in our new coordinate system,
we're seeing 3, 2.

00:11:03.390 --> 00:11:06.690
So that means 3 times our first
direction, which happens

00:11:06.690 --> 00:11:07.950
to be the v1 direction.

00:11:07.950 --> 00:11:09.490
It's no longer the horizontal
direction.

00:11:09.490 --> 00:11:12.680
It's the v1 direction.

00:11:12.680 --> 00:11:18.500
We're going 1, 2, and then
we go 3, like that.

00:11:18.500 --> 00:11:21.470
And then we're going to go
2 in the v2 direction.

00:11:21.470 --> 00:11:26.260
So we're going to go 1,
2 in the v2 direction.

00:11:26.260 --> 00:11:29.110
And so our point is going
to be right there.

00:11:29.110 --> 00:11:30.250
You could imagine
going like this.

00:11:30.250 --> 00:11:34.390
You go 3 in the v1 direction,
and then you go 1, 2 in the v2

00:11:34.390 --> 00:11:35.900
direction and you get
to our point.

00:11:35.900 --> 00:11:38.980
Or you could go, kind of in
your v2 direction and then

00:11:38.980 --> 00:11:41.250
your v1 direction, but either
way you're going to get to

00:11:41.250 --> 00:11:42.680
your original point.

00:11:42.680 --> 00:11:46.570
So that vector, or the position
specified by the

00:11:46.570 --> 00:11:50.930
vector, 8, 7, could just as
easily be specified in our new

00:11:50.930 --> 00:11:54.180
coordinate system by the
coordinates 3, 2.

00:11:54.180 --> 00:11:59.980
Because we're saying 3 times v1,
and then, plus 3 times v1.

00:11:59.980 --> 00:12:01.180
It takes us in this direction.

00:12:01.180 --> 00:12:04.850
We're going 3 notches in the v1
direction, and then we're

00:12:04.850 --> 00:12:07.570
going 2 notches in
the v2 direction.

00:12:07.570 --> 00:12:13.430
And so that's why these are
called coordinates.

00:12:13.430 --> 00:12:16.690
You're literally saying how
many spaces in the v1

00:12:16.690 --> 00:12:19.430
direction to go, and then how
many spaces in the v2

00:12:19.430 --> 00:12:20.250
direction to go.

00:12:20.250 --> 00:12:24.460
But this might, I guess, lead
you to the obvious question.

00:12:24.460 --> 00:12:27.280
Why haven't we been using
the coordinates before?

00:12:27.280 --> 00:12:29.640
Like I might have been
saying all along.

00:12:29.640 --> 00:12:32.470
I've been saying all along.

00:12:32.470 --> 00:12:37.660
Let's say I have some vector
lowercase b, that is equal to,

00:12:37.660 --> 00:12:39.380
I don't know, let's say it's
equal to-- I'll do it in an

00:12:39.380 --> 00:12:41.640
R2, just because it's easy to
visualize-- let's say it's

00:12:41.640 --> 00:12:46.630
equal to 3 minus 1.

00:12:46.630 --> 00:12:50.240
If we were to graph
it it would look

00:12:50.240 --> 00:12:52.480
something like this.

00:12:52.480 --> 00:12:57.490
We would go, 1, 2, 3, and
then we would go down 1.

00:12:57.490 --> 00:12:59.220
So it would look something
like this.

00:12:59.220 --> 00:13:00.970
It would specify this point.

00:13:00.970 --> 00:13:04.310
But why have we been calling 3
and negative 1 coordinates?

00:13:04.310 --> 00:13:07.110
Why have we been calling 3
and minus 1 coordinates?

00:13:07.110 --> 00:13:08.630
We've been doing it
well before we

00:13:08.630 --> 00:13:09.490
learned linear algebra.

00:13:09.490 --> 00:13:12.440
We called these coordinates all
the way from when we first

00:13:12.440 --> 00:13:13.370
learned how to graph.

00:13:13.370 --> 00:13:16.680
Why are we calling those
coordinates?

00:13:16.680 --> 00:13:19.820
Or how does this meeting of
coordinates relate to these

00:13:19.820 --> 00:13:22.030
coordinates with respect
to a basis?

00:13:22.030 --> 00:13:24.560
Well, these are coordinates
with respect to a basis.

00:13:24.560 --> 00:13:26.100
These are actually coordinates
with respect to

00:13:26.100 --> 00:13:27.670
the standard basis.

00:13:27.670 --> 00:13:30.630
If you imagine, let's see,
the standard basis in

00:13:30.630 --> 00:13:31.680
R2 looks like this.

00:13:31.680 --> 00:13:36.530
We could have e1, which
is 1, 0, and we have

00:13:36.530 --> 00:13:40.680
e2, which is 0, 1.

00:13:40.680 --> 00:13:43.755
This is just the convention for
the standard basis in R2.

00:13:43.755 --> 00:13:53.100
And so we could say s is equal
to the set of e1 and e2.

00:13:53.100 --> 00:14:06.055
Then we say that s is the
standard basis for R2.

00:14:06.055 --> 00:14:09.080
And it's a standard basis
because these two guys are

00:14:09.080 --> 00:14:10.180
orthogonal.

00:14:10.180 --> 00:14:12.470
This is 1 in the horizontal
direction.

00:14:12.470 --> 00:14:14.130
This is 1 in the vertical
direction.

00:14:14.130 --> 00:14:22.130
And any vector in R2-- let's say
I have some vector x, y in

00:14:22.130 --> 00:14:32.440
R2-- it's going to be equal to
x times e1, plus y times e2.

00:14:32.440 --> 00:14:39.900
So we can say that if you want
to write some vector x, y, if

00:14:39.900 --> 00:14:43.600
you wanted to write with respect
to this standard basis

00:14:43.600 --> 00:14:53.770
right here, it's going to be
equal to the coordinates by

00:14:53.770 --> 00:14:57.270
the definition that we did
earlier in this video of the

00:14:57.270 --> 00:14:58.600
basis vectors right there.

00:14:58.600 --> 00:15:01.250
Or these weights on our e1's and
e2's, so it's going to be

00:15:01.250 --> 00:15:04.010
equal to, well the weight
there is x and the

00:15:04.010 --> 00:15:05.780
weight here is y.

00:15:05.780 --> 00:15:08.410
So these coordinates that we've
been talking about from

00:15:08.410 --> 00:15:10.820
the get-go, these are definitely
coordinates.

00:15:10.820 --> 00:15:13.400
They're consistent with our
definition of coordinates in

00:15:13.400 --> 00:15:14.300
this video.

00:15:14.300 --> 00:15:16.180
But we can maybe be a little
bit more precise.

00:15:16.180 --> 00:15:25.930
We can now call them the
coordinates with respect to

00:15:25.930 --> 00:15:27.480
the standard basis.

00:15:33.950 --> 00:15:37.700
Or we could call them, we could
call these right here

00:15:37.700 --> 00:15:38.950
the standard coordinates.

00:15:41.750 --> 00:15:42.910
I just wanted to
point this out.

00:15:42.910 --> 00:15:46.600
This might be almost trivially
simple, or a bit obvious.

00:15:46.600 --> 00:15:49.370
But I just wanted to show that
our old usage of the word

00:15:49.370 --> 00:15:52.210
coordinates was not inconsistent
with this new

00:15:52.210 --> 00:15:58.560
definition of coordinates as
being the weights on some

00:15:58.560 --> 00:15:59.610
basis vectors.

00:15:59.610 --> 00:16:02.290
Because even in our old
coordinates, or the old way we

00:16:02.290 --> 00:16:04.970
used them, these really
were weights on our

00:16:04.970 --> 00:16:07.270
standard basis vectors.


Least squares examples

WEBVTT
Kind: captions
Language: en

00:00:00.570 --> 00:00:03.800
So I've got three lines in R2,
and I want to find their

00:00:03.800 --> 00:00:04.600
intersection.

00:00:04.600 --> 00:00:11.430
So the first one is 2x minus y
is equal to 2, the second one

00:00:11.430 --> 00:00:20.550
is x plus 2y is equal to 1, and
the third one is x plus y

00:00:20.550 --> 00:00:22.520
is equal to 4.

00:00:22.520 --> 00:00:24.790
So let's first just graph these,
just to have a visual

00:00:24.790 --> 00:00:27.040
representation of what
we're trying to do.

00:00:27.040 --> 00:00:30.380
So I like writing my lines in
y equals mx plus b form.

00:00:30.380 --> 00:00:32.040
So this top line becomes what?

00:00:32.040 --> 00:00:37.630
Minus y is equal to minus
2x, minus 2x plus 2.

00:00:37.630 --> 00:00:39.810
I just subtracted 2x
from both sides.

00:00:39.810 --> 00:00:44.140
Or we could write that y
is equal to 2x minus 2.

00:00:44.140 --> 00:00:45.900
That's the first line here.

00:00:45.900 --> 00:00:49.350
The second line-- I'm doing it
in green --we could write this

00:00:49.350 --> 00:00:55.630
as 2y is equal to minus x plus
1, or we could write that y is

00:00:55.630 --> 00:01:00.130
equal to minus 1/2 x plus 1/2.

00:01:00.130 --> 00:01:02.540
I just divided both
sides by 2.

00:01:02.540 --> 00:01:06.880
And this last line right here,
we could write this as y is

00:01:06.880 --> 00:01:08.760
equal to minus x plus 4.

00:01:08.760 --> 00:01:09.440
Go straight here.

00:01:09.440 --> 00:01:14.360
y is equal to minus x plus 4.

00:01:14.360 --> 00:01:17.570
Now let me graph these.

00:01:17.570 --> 00:01:22.100
Let me draw an axis,
that is my y-axis.

00:01:22.100 --> 00:01:24.740
I can call it my y-axis since
we're actually dealing with x

00:01:24.740 --> 00:01:27.590
and y's now.

00:01:27.590 --> 00:01:28.710
I'll do it slightly lighter.

00:01:28.710 --> 00:01:31.670
I'll do it in this
grey as well.

00:01:31.670 --> 00:01:34.470
I'd say that is my x-axis.

00:01:34.470 --> 00:01:35.790
Just like that.

00:01:35.790 --> 00:01:38.580
And this first guy is going
to be 2x minus 2.

00:01:38.580 --> 00:01:42.180
So it's y-intercept is going
to be at minus 2.

00:01:42.180 --> 00:01:43.920
It's going to have a slope of
2, so it's going to be a

00:01:43.920 --> 00:01:46.720
pretty steep line,
just like that.

00:01:46.720 --> 00:01:48.870
So that's that first line,
just like that.

00:01:48.870 --> 00:01:50.270
2x minus 2.

00:01:50.270 --> 00:01:54.780
The next line is 1/2,
minus 1/2, plus 1/2.

00:01:54.780 --> 00:01:58.290
So if we go plus 1/2, that's
right there, and then the

00:01:58.290 --> 00:02:00.760
slope is minus 1/2.

00:02:00.760 --> 00:02:05.290
So for every 2 we go
over, we go down 1.

00:02:05.290 --> 00:02:06.760
So it's going to be like that.

00:02:06.760 --> 00:02:08.289
It's actually going to
be orthogonal, right?

00:02:08.289 --> 00:02:10.355
Because it's the negative
inverse of this guy.

00:02:10.355 --> 00:02:14.820
So it's going to look
something like that.

00:02:14.820 --> 00:02:18.940
We draw it like that,
just like that.

00:02:18.940 --> 00:02:23.150
And this guy is minus
x plus 4.

00:02:23.150 --> 00:02:29.940
This is 1, 2, 3, 4, and you go
minus x, so for every 1 you go

00:02:29.940 --> 00:02:31.000
over, you go down 1.

00:02:31.000 --> 00:02:34.290
So this other line is going to
look something like this.

00:02:34.290 --> 00:02:36.540
This other line, this last
line, is going to look

00:02:36.540 --> 00:02:37.610
something like that.

00:02:37.610 --> 00:02:38.680
Just like that.

00:02:38.680 --> 00:02:41.340
Now, I said at the beginning of
this video that I want to

00:02:41.340 --> 00:02:43.610
find the intersection of
these three lines.

00:02:43.610 --> 00:02:45.570
But notice, there is
no intersection

00:02:45.570 --> 00:02:46.880
of these three lines.

00:02:46.880 --> 00:02:49.330
They all intersect the other
two, but they don't all

00:02:49.330 --> 00:02:51.440
intersect each other
in one point.

00:02:51.440 --> 00:02:54.000
We can kind of call the system
as being overdetermined.

00:02:54.000 --> 00:02:55.500
We've overconstrained it.

00:02:55.500 --> 00:02:58.740
There is no intersection of
all three of these points.

00:02:58.740 --> 00:03:01.360
So if I were to actually try to
solve this system, I would

00:03:01.360 --> 00:03:02.850
find no solution.

00:03:02.850 --> 00:03:05.070
And to say that this has no
solution is equivalent to

00:03:05.070 --> 00:03:08.170
saying that this
matrix, or this

00:03:08.170 --> 00:03:09.290
equation has no solutions.

00:03:09.290 --> 00:03:09.930
Let me write this.

00:03:09.930 --> 00:03:12.690
I'm just going to rewrite
the system like this.

00:03:12.690 --> 00:03:18.450
This is equivalent to the
matrix, let me make sure I get

00:03:18.450 --> 00:03:26.660
this right, the matrix times
the vector xy is equal

00:03:26.660 --> 00:03:31.610
to 2, 1, and 4.

00:03:31.610 --> 00:03:36.440
And so, this first equation is
2 times x, minus 1 times y.

00:03:36.440 --> 00:03:38.850
So it's 2 and minus 1.

00:03:38.850 --> 00:03:41.340
2 times x minus 1 times
y is equal to 2.

00:03:41.340 --> 00:03:43.410
That's that first equation
over there.

00:03:43.410 --> 00:03:46.050
The second equation-- actually
I could even, well I won't

00:03:46.050 --> 00:03:48.490
color code it, that'll take
forever --that's 1 times x

00:03:48.490 --> 00:03:50.960
plus 2 times y is equal to 1.

00:03:50.960 --> 00:03:53.650
And then we have x plus
y is equal to 4.

00:03:53.650 --> 00:03:56.770
This system and this equation,
this system right here, these

00:03:56.770 --> 00:03:57.760
are equivalent.

00:03:57.760 --> 00:03:59.410
Now, this isn't going to
have any solution.

00:03:59.410 --> 00:04:01.330
You could try to find
a solution to this.

00:04:01.330 --> 00:04:04.270
You could create an augmented
matrix, put it in reduced row

00:04:04.270 --> 00:04:04.960
echelon form.

00:04:04.960 --> 00:04:07.220
But there is no intersection
to the three things.

00:04:07.220 --> 00:04:13.690
So you're not going to find
a solution to A times some

00:04:13.690 --> 00:04:17.550
vector-- we could call this some
vector x --is equal to B.

00:04:17.550 --> 00:04:20.480
Or another way to say it, is
that B is not in the column

00:04:20.480 --> 00:04:23.690
space of this matrix
right here.

00:04:23.690 --> 00:04:26.370
Now, we learned in the last
video that sure, we can't find

00:04:26.370 --> 00:04:28.590
a solution to Ax equals B.

00:04:28.590 --> 00:04:31.600
Ax equals B has no solution.

00:04:34.110 --> 00:04:36.890
We see it graphically here;
these lines don't intersect

00:04:36.890 --> 00:04:37.410
with each other.

00:04:37.410 --> 00:04:39.820
And you could prove it for
yourself algebraically by

00:04:39.820 --> 00:04:41.076
trying to find a
solution here.

00:04:41.076 --> 00:04:43.680
You'll end up with
a 0 equals 1.

00:04:43.680 --> 00:04:46.300
But we can almost get
there by finding a

00:04:46.300 --> 00:04:47.780
least squares solution.

00:04:47.780 --> 00:04:49.970
And we find a least squares
solution if we multiply both

00:04:49.970 --> 00:04:51.820
sides by A transpose.

00:04:51.820 --> 00:04:56.890
We know that A transpose times
A times our least squares

00:04:56.890 --> 00:05:02.330
solution is going to be equal
to A transpose times B.

00:05:02.330 --> 00:05:06.180
So at least we can find the
closest fit for our solution.

00:05:06.180 --> 00:05:08.160
So let's find the vector
x this is our

00:05:08.160 --> 00:05:10.310
least squares solution.

00:05:10.310 --> 00:05:13.120
So what is A transpose
times A?

00:05:13.120 --> 00:05:16.730
A transpose looks like
this, you'll have 2

00:05:16.730 --> 00:05:20.760
minus 1, 1, 2, 1, 1.

00:05:20.760 --> 00:05:22.740
That is A transpose.

00:05:22.740 --> 00:05:26.290
And then of course A is
just this thing: 2

00:05:26.290 --> 00:05:30.210
minus 1, 1, 1, 1, 1.

00:05:30.210 --> 00:05:35.760
So A transpose A is going to be
equal to-- We have a 2 by 3

00:05:35.760 --> 00:05:40.620
times a 3 by 2 matrix, so it's
going to be a 2 by 2 matrix.

00:05:40.620 --> 00:05:43.240
So it's going to be
a 2 by 2 matrix.

00:05:43.240 --> 00:05:47.660
So what do we get, we get 2
times 2 which is 4, plus 1

00:05:47.660 --> 00:05:49.620
times 1, plus 1 times 1.

00:05:49.620 --> 00:05:51.800
So it's 4 plus 1 plus 1.

00:05:51.800 --> 00:05:54.620
So that's equal to 6.

00:05:54.620 --> 00:05:59.770
And then we have 2 times minus
1, which is minus 2, plus 1

00:05:59.770 --> 00:06:02.470
times 2, so those cancel out.

00:06:02.470 --> 00:06:05.820
Minus 2 plus 2 is 0,
plus 1 times 1.

00:06:05.820 --> 00:06:08.070
That's just going to be 1.

00:06:08.070 --> 00:06:12.680
And then we get minus 1 times
2, which is minus 2, plus 2

00:06:12.680 --> 00:06:14.470
times 1, which is 2.

00:06:14.470 --> 00:06:18.540
So the minus 2 plus 2 is 0, plus
1 times 1, so we get a 1.

00:06:18.540 --> 00:06:21.670
And then finally, we get minus
1 times minus 1, which is

00:06:21.670 --> 00:06:22.710
positive 1.

00:06:22.710 --> 00:06:25.870
Plus 2 times 2, which is
4, so we're now at 5.

00:06:25.870 --> 00:06:28.880
Plus 1 times 1, so this
is going to be 6.

00:06:28.880 --> 00:06:31.950
So this is A transpose A.

00:06:31.950 --> 00:06:36.840
Now, what is A transpose
times B?

00:06:36.840 --> 00:06:44.190
A transpose is 2, 1,
1, minus 1, 2, 1.

00:06:44.190 --> 00:06:49.200
And then B is just the 3 by 1
vector, the vector that's a

00:06:49.200 --> 00:06:52.680
member of R3, 2, 1, 4.

00:06:52.680 --> 00:06:55.470
So what is this going
to be equal to?

00:06:55.470 --> 00:07:02.770
This is going to be equal
to-- We get a 3.

00:07:02.770 --> 00:07:07.300
Sorry, this is a 2 by
3 times a 3 by 1.

00:07:07.300 --> 00:07:10.640
We're going to get
a 2 by 1 vector.

00:07:10.640 --> 00:07:12.620
You get a 2 by 1 vector here.

00:07:12.620 --> 00:07:19.330
So 2 times 2 is 4, plus 1
times 1, so that's plus

00:07:19.330 --> 00:07:21.010
1, so that's 5.

00:07:21.010 --> 00:07:22.884
Plus-- let me actually write it
down, I'm going to make a

00:07:22.884 --> 00:07:26.940
careless mistake --2 times 2,
which is 4, plus 1 times 1,

00:07:26.940 --> 00:07:30.370
which is 1, plus 1 times
4, which is 4.

00:07:30.370 --> 00:07:34.210
Then here, you have minus 1
times 2, which is minus 2, 2

00:07:34.210 --> 00:07:39.640
times 1, which is 2, plus
1 times 4, which is 4.

00:07:39.640 --> 00:07:45.790
So A transpose times B is
equal to 9, and this

00:07:45.790 --> 00:07:48.090
is going to be 4.

00:07:48.090 --> 00:07:53.970
So we can rewrite this guy right
here as the matrix A

00:07:53.970 --> 00:07:54.560
transpose A.

00:07:54.560 --> 00:08:01.630
Which is just 6, 1, 1, 6 times
my least squares solution-- so

00:08:01.630 --> 00:08:05.620
this is actually going to be in
the column space of A --is

00:08:05.620 --> 00:08:12.510
equal to A transpose times B,
which is just the vector 9 4.

00:08:12.510 --> 00:08:14.890
And this'll be a little bit more
straightforward to find a

00:08:14.890 --> 00:08:15.530
solution for.

00:08:15.530 --> 00:08:16.740
In fact, there will
be a solution.

00:08:16.740 --> 00:08:18.160
We proved it in the
last video.

00:08:18.160 --> 00:08:20.750
So to find a solution, let's
create our little augmented

00:08:20.750 --> 00:08:24.450
matrix: 6, 1, augmented
with a 9.

00:08:24.450 --> 00:08:27.850
You have the 4 there,
you get a 1 and a 6.

00:08:27.850 --> 00:08:29.630
Just like that.

00:08:29.630 --> 00:08:32.860
Let's put the left hand side in
reduced row echelon form.

00:08:32.860 --> 00:08:37.200
Actually, first I'm going
to swap these two rows.

00:08:37.200 --> 00:08:40.150
That's my first row operation
that I choose to do, just

00:08:40.150 --> 00:08:41.520
because I like to have
that 1 there.

00:08:41.520 --> 00:08:43.230
It's a nice pivot entry.

00:08:43.230 --> 00:08:51.640
So then it goes to 1,
6, 4, and 6, 1, 9.

00:08:51.640 --> 00:08:56.180
And then let me replace my
second row with the second row

00:08:56.180 --> 00:08:58.180
minus 6 times the first row.

00:08:58.180 --> 00:09:01.250
So I'm going to keep my
first row the same.

00:09:01.250 --> 00:09:05.210
So I have 1, 6, and 4.

00:09:05.210 --> 00:09:09.190
And then my second row, I'm
going to replace my second row

00:09:09.190 --> 00:09:12.610
with a second row minus
6 times the first row.

00:09:12.610 --> 00:09:15.480
So 6 minus 6 is 0.

00:09:15.480 --> 00:09:23.890
1 minus 6 times 6, that's 1
minus 36, so that's minus 35.

00:09:23.890 --> 00:09:29.250
And then a 9 minus 6 times
4, is 9 minus 24.

00:09:29.250 --> 00:09:30.480
This one always gets
me in trouble.

00:09:30.480 --> 00:09:36.510
So 9 minus 24, that's the
negative of 24 minus 9, so

00:09:36.510 --> 00:09:41.460
that is minus 15.

00:09:41.460 --> 00:09:43.320
Let me make sure I didn't
make a careless mistake.

00:09:43.320 --> 00:09:51.930
1 minus 36 is minus 35, 9
minus 24, is minus 15.

00:09:51.930 --> 00:09:54.710
So that's what I get
right there.

00:09:54.710 --> 00:09:59.260
And then let me go to the right
now, so let me let me

00:09:59.260 --> 00:10:04.020
divide this row right here, let
me divide it by minus 35.

00:10:04.020 --> 00:10:05.250
So I'm going to keep my
first row the same.

00:10:05.250 --> 00:10:06.980
1, 6, and 4.

00:10:06.980 --> 00:10:08.190
And then this guy right
here, I'm going to

00:10:08.190 --> 00:10:10.300
divide by minus 35.

00:10:10.300 --> 00:10:16.810
So I'm going to get 0, 1, and
then minus 15 over minus 35.

00:10:16.810 --> 00:10:21.940
That's 15 over 35, or
that's 3 over 7, so

00:10:21.940 --> 00:10:25.620
that is equal to 3/7.

00:10:25.620 --> 00:10:29.300
And then let me just put this
in complete reduced row

00:10:29.300 --> 00:10:30.110
echelon form.

00:10:30.110 --> 00:10:31.840
That'll be nice.

00:10:31.840 --> 00:10:33.830
Let me keep my second
row the same.

00:10:33.830 --> 00:10:39.280
So my second row is
0, 1, and 3/7.

00:10:39.280 --> 00:10:42.710
And then my first row, I'm going
to replace it with my

00:10:42.710 --> 00:10:46.180
first row minus 6 times
my second row.

00:10:46.180 --> 00:10:51.590
So 1 minus 6 times 0 is 1, 6
minus 6 times 1 is 0, and then

00:10:51.590 --> 00:10:55.610
we have 4 minus 6 times 3/7.

00:10:55.610 --> 00:10:57.990
So 4 is 28/7.

00:10:57.990 --> 00:10:59.360
Let me write it up here.

00:10:59.360 --> 00:11:06.690
So we have 4, which is 28/7,
minus 6 times 3/7, so minus

00:11:06.690 --> 00:11:07.980
18/7, right?

00:11:07.980 --> 00:11:10.220
That's 6 times 3/7.

00:11:10.220 --> 00:11:18.120
So this is going to
be equal to 10/7.

00:11:18.120 --> 00:11:20.590
And just like that, I've solved
this new equation.

00:11:20.590 --> 00:11:24.010
So you could say that-- Let
me write it this way.

00:11:24.010 --> 00:11:28.410
We could write that x star--
This is going to the first

00:11:28.410 --> 00:11:30.280
entry of x star, which
we could call x,

00:11:30.280 --> 00:11:31.300
is going to be 10/7.

00:11:31.300 --> 00:11:33.010
Let me write this.

00:11:33.010 --> 00:11:42.130
x star, the solution is going
to be 10/7 and 3/7.

00:11:42.130 --> 00:11:45.390
So I'm saying that if you take
x is equal to 10/7 and y is

00:11:45.390 --> 00:11:48.210
equal to 3/7, you're going
to get as close to

00:11:48.210 --> 00:11:50.490
a solution as possible.

00:11:50.490 --> 00:11:52.240
Let's see what that looks
like visually?

00:11:52.240 --> 00:11:54.050
What is 10/7?

00:11:54.050 --> 00:12:05.330
Let me write this down. x star
is equal to 10/7 and 3/7.

00:12:05.330 --> 00:12:08.245
We're saying the closest-- Our
least squares solution is x is

00:12:08.245 --> 00:12:13.850
equal to 10/7, so x is
a little over one.

00:12:13.850 --> 00:12:17.320
And then y is going to be 3/7,
a little less than 1/2.

00:12:17.320 --> 00:12:20.720
So our least squares solution
is going to be

00:12:20.720 --> 00:12:22.860
this one, right there.

00:12:22.860 --> 00:12:30.510
And so this, when you put this
value for x, when you put x is

00:12:30.510 --> 00:12:39.080
equal to 10/7 and y is equal
to 3/7, you're going to

00:12:39.080 --> 00:12:42.340
minimize the collective squares
of the distances

00:12:42.340 --> 00:12:44.600
between all of these guys.

00:12:44.600 --> 00:12:47.300
I drew this a little bit
too small to show that.

00:12:47.300 --> 00:12:49.790
But let's actually figure out
what our least, what our

00:12:49.790 --> 00:12:53.670
minimized difference is.

00:12:53.670 --> 00:12:57.660
Remember, the whole point of
the is to minimize the

00:12:57.660 --> 00:13:02.370
distance between Ax star, is
to minimize the distance

00:13:02.370 --> 00:13:05.450
between Ax star and B.

00:13:05.450 --> 00:13:08.660
Or between B and Ax star.

00:13:08.660 --> 00:13:11.366
Now, what was Ax
star equal to?

00:13:11.366 --> 00:13:20.560
Ax star was equal to 9/4.

00:13:20.560 --> 00:13:23.250
So this right here.

00:13:23.250 --> 00:13:25.730
Sorry, Ax star, that's not
equal to 9/4, that's A

00:13:25.730 --> 00:13:28.040
transpose Ax star
is equal to 9/4.

00:13:28.040 --> 00:13:32.360
Ax star is our original matrix
A, which is this one right

00:13:32.360 --> 00:13:36.590
here, so 2-- Let me write
it down here.

00:13:36.590 --> 00:13:37.880
I know it's off the
page right now.

00:13:37.880 --> 00:13:43.410
So our original matrix A was
2, minus 1, 1, 2, 1, 1.

00:13:43.410 --> 00:13:47.150
That was our original matrix
A right there.

00:13:47.150 --> 00:13:50.010
And then our x star,
we were able to

00:13:50.010 --> 00:13:54.710
determine, is 10/7 and 3/7.

00:13:54.710 --> 00:13:58.240
So Ax star is going to be
this product, so what

00:13:58.240 --> 00:13:59.740
does it equal to?

00:13:59.740 --> 00:14:05.120
It is equal to, it is going
to be a 3 by 1 matrix.

00:14:05.120 --> 00:14:11.460
So we got 2 times 10/7, which
is 20/7, minus 1 times

00:14:11.460 --> 00:14:16.010
3/7, so minus 3/7.

00:14:16.010 --> 00:14:19.830
And then we get, let me see,
yep, that's 10 minus 3/7, and

00:14:19.830 --> 00:14:28.210
we have 10/7, minus 2 times
3/7, so minus 6/7.

00:14:28.210 --> 00:14:30.490
Or plus, sorry, this
is a plus.

00:14:30.490 --> 00:14:30.850
Right?

00:14:30.850 --> 00:14:35.280
2 times 10/7 is 20/7 minus 1
times 3/7, then we have 1

00:14:35.280 --> 00:14:37.775
times 10/7 plus 2 times 3/7.

00:14:37.775 --> 00:14:44.180
And then we have
10/7 plus 3/7.

00:14:44.180 --> 00:14:48.600
So Ax, so this is A and x
star, our least squares

00:14:48.600 --> 00:14:53.450
approximation for x, is
equal to what is this?

00:14:53.450 --> 00:15:02.590
This is 17/7, this is 16/7,
and this is 13/7.

00:15:02.590 --> 00:15:04.750
We want to find out with this
minimum distance is.

00:15:04.750 --> 00:15:08.290
So, let's see, this is going
to be this thing.

00:15:08.290 --> 00:15:17.770
So 17/7, 16/7, and 13/7
minus our original B.

00:15:17.770 --> 00:15:23.690
Our original B was
2, 1, and 4.

00:15:23.690 --> 00:15:26.790
So I'm claiming that my solution
that we just found,

00:15:26.790 --> 00:15:28.530
this minimizes this distance.

00:15:28.530 --> 00:15:32.900
Because this is the projection
of B onto the

00:15:32.900 --> 00:15:33.980
column space of A.

00:15:33.980 --> 00:15:34.970
We saw that before.

00:15:34.970 --> 00:15:37.910
So our B we see all the
way up here is 2, 1,

00:15:37.910 --> 00:15:41.850
4, just like that.

00:15:41.850 --> 00:15:46.850
So if we take the length of
this-- let me switch colors

00:15:46.850 --> 00:15:49.450
--this is equal to the length.

00:15:49.450 --> 00:15:51.530
Let me write all of us
in sevenths-- I'll

00:15:51.530 --> 00:15:53.340
just do it in my head.

00:15:53.340 --> 00:15:55.280
I don't want to waste
too much time.

00:15:55.280 --> 00:16:00.150
So this is 17/7 minus
14/7, right?

00:16:00.150 --> 00:16:03.690
2 is 14/7, so this is
going to be 3/7.

00:16:03.690 --> 00:16:09.980
Then it's 16/7 minus
7/7, so that's 9/7.

00:16:09.980 --> 00:16:15.210
And then we have 13/7 minus
28/7, so that is

00:16:15.210 --> 00:16:19.150
minus 15 over 7.

00:16:19.150 --> 00:16:23.280
So this is the vector that
separates the B that was not

00:16:23.280 --> 00:16:27.970
in my column space of A from
the projection of B.

00:16:27.970 --> 00:16:31.740
And if we find it's length,
it's length is going to be

00:16:31.740 --> 00:16:36.710
equal to-- Let's find the square
of it's length first.

00:16:36.710 --> 00:16:40.900
The square of its length is
going to be 3/7 squared, so

00:16:40.900 --> 00:16:53.310
that is 9/49, plus 9/7 squared,
which is 81/49, plus

00:16:53.310 --> 00:16:55.470
minus 15/7 squared.

00:16:55.470 --> 00:16:56.650
What's 15 squared?

00:16:56.650 --> 00:16:59.860
15 squared is 225, I think.

00:16:59.860 --> 00:17:01.440
Let me make sure.

00:17:01.440 --> 00:17:02.920
I'm prone to careless
mistakes.

00:17:02.920 --> 00:17:08.659
5 times 5 is 25, 1 times
5, this is 75, and

00:17:08.659 --> 00:17:10.180
then I have a 150.

00:17:10.180 --> 00:17:13.260
Yep, 225.

00:17:13.260 --> 00:17:18.109
So plus 225/49.

00:17:18.109 --> 00:17:24.960
Which is equal to-- So 9 plus 81
is 90, and then so 225 plus

00:17:24.960 --> 00:17:30.510
90, to get a 5, was a 315.

00:17:30.510 --> 00:17:35.640
So this is equal to 315/49.

00:17:35.640 --> 00:17:38.690
Or, if we actually wanted the
difference, that's going to

00:17:38.690 --> 00:17:41.530
square root of this.

00:17:41.530 --> 00:17:44.130
So if we take just the regular
distance, that's equal to the

00:17:44.130 --> 00:17:46.180
square root of that.

00:17:46.180 --> 00:17:56.250
So it's equal to the square
root of 315 over 7.

00:17:56.250 --> 00:17:59.310
And square root of 315,
it looks like that is

00:17:59.310 --> 00:18:00.380
simplifiable.

00:18:00.380 --> 00:18:01.740
Does 9 go into it?

00:18:01.740 --> 00:18:04.880
Looks like 9 goes into it,
maybe 35 five times?

00:18:04.880 --> 00:18:09.440
So it would be what, 3 square
roots of 35 over 7.

00:18:09.440 --> 00:18:12.680
So that is just a measure.

00:18:12.680 --> 00:18:14.500
Let me put it this way, you're
not going to be able to find

00:18:14.500 --> 00:18:19.150
any member of R2, any values
of x and y, that's going to

00:18:19.150 --> 00:18:22.660
give you a smaller value than
this when you find the

00:18:22.660 --> 00:18:25.260
distance between it's solution
and the solution you were

00:18:25.260 --> 00:18:27.040
trying to get to.

00:18:27.040 --> 00:18:32.300
So this, based on our least
squares solution, is the best

00:18:32.300 --> 00:18:36.040
estimate you're going to get.
x is equal to 10/7,

00:18:36.040 --> 00:18:38.700
y is equal to 3/7.

00:18:38.700 --> 00:18:42.560
A little bit right,
just like that.

00:18:42.560 --> 00:18:44.180
Anyway, hopefully you found
that useful, and you're

00:18:44.180 --> 00:18:46.620
starting to appreciate that the
least squares solution is

00:18:46.620 --> 00:18:48.240
pretty useful.


Another least squares example

WEBVTT
Kind: captions
Language: en

00:00:00.490 --> 00:00:02.840
So I've got four Cartesian
coordinates here.

00:00:02.840 --> 00:00:05.770
This first one is minus 1, 0.

00:00:05.770 --> 00:00:07.440
I tried to draw them
ahead of time.

00:00:07.440 --> 00:00:10.230
So minus 1, 0 is this
point right there.

00:00:10.230 --> 00:00:12.510
Doing this in these
new colors.

00:00:12.510 --> 00:00:17.360
The next point is
a 0, 1, which is

00:00:17.360 --> 00:00:19.000
that point right there.

00:00:19.000 --> 00:00:23.840
Then the next point is
1, 2, which is that

00:00:23.840 --> 00:00:25.320
point right up there.

00:00:25.320 --> 00:00:32.240
And then the last point is 2, 1,
which is that point there.

00:00:32.240 --> 00:00:36.310
Now my goal in this video is to
find some line, y equals mx

00:00:36.310 --> 00:00:40.040
plus v, that goes through
these points.

00:00:40.040 --> 00:00:41.690
Now the first thing I'd say
is, hey Sal, there is not

00:00:41.690 --> 00:00:43.670
going to be any line that goes
through these points, and you

00:00:43.670 --> 00:00:44.920
can see that immediately.

00:00:44.920 --> 00:00:46.550
You could find a line that
maybe goes through these

00:00:46.550 --> 00:00:47.630
points, but it's not
going to go through

00:00:47.630 --> 00:00:48.540
this point over here.

00:00:48.540 --> 00:00:50.960
If you try to make a line to
goes through these two points,

00:00:50.960 --> 00:00:52.580
it's not going to go through
those points there.

00:00:52.580 --> 00:00:55.820
So you're not going to be able
to find a solution that goes

00:00:55.820 --> 00:00:58.180
through those points.

00:00:58.180 --> 00:01:00.580
Let's set up the equation that
we know we can't find the

00:01:00.580 --> 00:01:03.350
solution to and maybe we can
use our least squares

00:01:03.350 --> 00:01:07.690
approximation to find a line
that almost goes through all

00:01:07.690 --> 00:01:08.370
these points.

00:01:08.370 --> 00:01:10.780
Or it's at least the best
approximation for a line that

00:01:10.780 --> 00:01:12.420
goes through those points.

00:01:12.420 --> 00:01:16.200
So this first one, I can
express my line, y

00:01:16.200 --> 00:01:16.980
equals mx plus b.

00:01:16.980 --> 00:01:22.430
Let me just express it as f of
x is equal to mx plus b, or y

00:01:22.430 --> 00:01:23.740
is equal to f of x.

00:01:23.740 --> 00:01:25.480
We can write it that way.

00:01:25.480 --> 00:01:28.710
So our first point right there
-- let me do it in that color,

00:01:28.710 --> 00:01:34.340
that orange -- that tells us
that f of minus 1, which is

00:01:34.340 --> 00:01:37.760
equal to m times -- let me just
write this way -- minus 1

00:01:37.760 --> 00:01:42.160
times m, it's minus m plus
b, that that is going

00:01:42.160 --> 00:01:44.180
to be equal to 0.

00:01:44.180 --> 00:01:46.550
That's what that first
equation tells us.

00:01:46.550 --> 00:01:53.530
The second equation tells us
that f of 0, which is equal to

00:01:53.530 --> 00:02:00.210
0 times m, which is just
0 plus b is equal to 1.

00:02:00.210 --> 00:02:01.530
f of 0 is 1.

00:02:04.080 --> 00:02:06.050
This is f of x.

00:02:06.050 --> 00:02:09.380
The next one -- let me do it in
this yellow color -- tells

00:02:09.380 --> 00:02:18.160
us that f of 1, which is equal
to 1 times m, or just m, plus

00:02:18.160 --> 00:02:21.400
b, is going to be equal to 2.

00:02:21.400 --> 00:02:27.820
And then this last one down
here tells us that f of 2,

00:02:27.820 --> 00:02:33.420
which is of course 2 times m
plus b, that that is going to

00:02:33.420 --> 00:02:35.150
be equal to 1.

00:02:35.150 --> 00:02:36.260
These are the constraints.

00:02:36.260 --> 00:02:38.570
If we assume that our line can
go through all of these

00:02:38.570 --> 00:02:41.690
points, then all of these
things must be true.

00:02:41.690 --> 00:02:44.680
Now you could immediately, if
you wish, try to solve this

00:02:44.680 --> 00:02:48.050
equation, but you'll find that
you won't find a solution.

00:02:48.050 --> 00:02:51.420
We want to find some m's
and b's that satisfy

00:02:51.420 --> 00:02:53.570
all of these equations.

00:02:53.570 --> 00:02:56.820
Or another way of writing this
-- We want to write it as a

00:02:56.820 --> 00:02:59.600
matrix vector or a
matrix equation .

00:02:59.600 --> 00:03:01.530
We could write it like this.

00:03:01.530 --> 00:03:19.900
Minus 1, 1, 0, 1, 1, 1, 2, 1,
times the vector mv has got to

00:03:19.900 --> 00:03:24.340
be equal to the vector
0, 1, 2, 1.

00:03:27.770 --> 00:03:31.330
These two systems, this system
and this system right here,

00:03:31.330 --> 00:03:32.780
are equivalent statements,
right?

00:03:32.780 --> 00:03:37.060
Minus 1 times m plus 1 times b
has got to be equal to that 0.

00:03:37.060 --> 00:03:40.390
0 times m plus 1 times b has
got to be equal to that 1

00:03:40.390 --> 00:03:42.710
That's equivalent to that
statement right here.

00:03:42.710 --> 00:03:44.630
And this isn't going
to have a solution.

00:03:44.630 --> 00:03:47.680
The solution would have to go
through all of those points.

00:03:47.680 --> 00:03:51.870
So let's at least try to find
a least squares solution.

00:03:51.870 --> 00:03:57.880
So if we call this a, if we call
that x, and let's call

00:03:57.880 --> 00:04:07.780
this b, there is no solution
to ax is equal to b.

00:04:07.780 --> 00:04:10.250
Now maybe we can find a least
-- Well, we can definitely

00:04:10.250 --> 00:04:11.490
find a least squares solution.

00:04:11.490 --> 00:04:20.750
So let's find our least squares
solution such that a

00:04:20.750 --> 00:04:26.870
transpose a times our least
squares solution is equal to a

00:04:26.870 --> 00:04:28.900
transpose times b.

00:04:28.900 --> 00:04:30.800
Our least squares solution
is the one that

00:04:30.800 --> 00:04:32.020
satisfies this equation.

00:04:32.020 --> 00:04:34.400
We proved it two videos ago.

00:04:34.400 --> 00:04:36.550
So let's figure out what a
transpose a is and what a

00:04:36.550 --> 00:04:39.400
transpose b is, and
then we can solve.

00:04:39.400 --> 00:04:43.700
So a transpose will
look like this.

00:04:43.700 --> 00:04:51.670
b minus 1, 1, 0, 1, 1,
1, and then 2, 1.

00:04:51.670 --> 00:04:54.920
This first column becomes this
first row; this second column

00:04:54.920 --> 00:04:56.850
becomes this second row.

00:04:56.850 --> 00:04:59.270
So we're going to take the
product of a transpose and

00:04:59.270 --> 00:05:04.870
then a-- a is that thing right
there --minus 1, 0, 1, 2, and

00:05:04.870 --> 00:05:07.270
we just get a bunch of 1's.

00:05:07.270 --> 00:05:08.500
So what does this equal to?

00:05:08.500 --> 00:05:11.250
We have a 2 by 4
times a 4 by 2.

00:05:11.250 --> 00:05:15.860
So we're going to have
a 2 by 2 matrix.

00:05:15.860 --> 00:05:19.220
So this is going to be --
Let's do it this way.

00:05:19.220 --> 00:05:22.390
Well, we're going to have minus
1 times minus 1, which

00:05:22.390 --> 00:05:28.940
is 1, plus 0 times 0, which is
0 -- so we're at 1 right now

00:05:28.940 --> 00:05:30.520
--plus 1 times 1.

00:05:30.520 --> 00:05:34.230
So that's 1 plus the other
1 up there, so that's 2,

00:05:34.230 --> 00:05:36.180
plus 2 times 2.

00:05:36.180 --> 00:05:38.170
2 times 2 is 4, so we get 6.

00:05:41.630 --> 00:05:46.340
That's that row, dotted with
that column, was equal to 6.

00:05:46.340 --> 00:05:51.030
Now let's take this row dotted
with this column.

00:05:51.030 --> 00:05:54.143
So it's going to be negative 1
times 1, plus 0 times 1, so

00:05:54.143 --> 00:05:55.910
all of these guys times
1 plus each other.

00:05:55.910 --> 00:06:00.350
So minus 1 plus 0 plus 1 --
that's all 0's --plus 2.

00:06:00.350 --> 00:06:02.700
So it's going to get a 2.

00:06:02.700 --> 00:06:05.600
I just dotted that guy
with that guy.

00:06:05.600 --> 00:06:11.170
Now I need to take the dot of
this guy with this column.

00:06:11.170 --> 00:06:13.890
So it's just going to be 1 times
minus 1 plus 1 times 0

00:06:13.890 --> 00:06:15.580
plus 1 times 1 plus 1 times 2.

00:06:15.580 --> 00:06:18.090
Well, these are all 1 times
everything, so it's minus 1

00:06:18.090 --> 00:06:20.185
plus 0 plus 1, which
is 0 plus 2.

00:06:20.185 --> 00:06:22.160
It's going to be 2.

00:06:22.160 --> 00:06:24.300
And then finally -- Well.

00:06:24.300 --> 00:06:26.360
I mean, I think you see
some symmetry here.

00:06:26.360 --> 00:06:29.040
We're going to have to take the
dot of this guy and this

00:06:29.040 --> 00:06:30.340
guy over here.

00:06:30.340 --> 00:06:30.900
So what is that?

00:06:30.900 --> 00:06:33.930
That's 1 times 1, which is 1,
plus 1 times 1, which is 2,

00:06:33.930 --> 00:06:35.400
plus 1 times 1.

00:06:35.400 --> 00:06:38.450
So we're going to have 1
plus itself four times.

00:06:38.450 --> 00:06:40.900
So we're going to get that
it's equal to 4.

00:06:40.900 --> 00:06:44.650
So this is a transpose a.

00:06:44.650 --> 00:06:48.270
And let's figure out what a
transpose b looks like.

00:06:48.270 --> 00:06:49.970
Scroll down a little bit.

00:06:49.970 --> 00:06:55.220
So a transpose is this matrix
again-- let me switch colors

00:06:55.220 --> 00:06:57.900
--minus 1, 0, 1, 2.

00:06:57.900 --> 00:07:01.160
We get all of our 1's
just like that.

00:07:01.160 --> 00:07:07.530
And then the matrix
b is 0, 1, 2, 1.

00:07:10.730 --> 00:07:14.490
We have a 2 by 4 times a 4 by 1,
so we're just going to get

00:07:14.490 --> 00:07:16.890
a 2 by 1 matrix.

00:07:16.890 --> 00:07:21.510
So this is going to be equal
to a 2 by 1 matrix.

00:07:21.510 --> 00:07:25.030
We have here, let's see, minus 1
times 0 is 0, plus 0 times 1

00:07:25.030 --> 00:07:26.440
is still 0.

00:07:26.440 --> 00:07:30.100
Plus 1 times 2, which
is 2, plus 2 times

00:07:30.100 --> 00:07:32.590
1, which is 4, right?

00:07:32.590 --> 00:07:36.560
This is 2 plus 2, so it's going
to be 4 right there.

00:07:36.560 --> 00:07:41.510
And then we have 1 times 0, plus
1 times 2, plus-- So 1

00:07:41.510 --> 00:07:43.240
times all of these
guys added up.

00:07:43.240 --> 00:07:48.630
So 0 plus 1 is 1, 1 plus
2 is 3, 3 plus 1 is 4.

00:07:48.630 --> 00:07:52.400
So this right here
is a transpose b.

00:07:52.400 --> 00:07:56.800
So just like that, we know
that the least squares

00:07:56.800 --> 00:07:59.080
solution will be the solution
to this system.

00:07:59.080 --> 00:08:08.460
6, 2, 2, 4, times our least
squares solution, is going to

00:08:08.460 --> 00:08:11.830
be equal to 4, 4.

00:08:11.830 --> 00:08:13.300
Or we could write it this way.

00:08:13.300 --> 00:08:19.250
We could write it 6, 2, 2, 4,
times our least squares

00:08:19.250 --> 00:08:21.020
solution, which I'll write--
Remember, the

00:08:21.020 --> 00:08:22.310
first entry was m .

00:08:22.310 --> 00:08:23.620
I'll write it as m star.

00:08:23.620 --> 00:08:28.130
That's our least square m, and
this is our least square b, is

00:08:28.130 --> 00:08:31.220
equal to 4, 4.

00:08:31.220 --> 00:08:33.159
And I can do this as an
augmented matrix or I could

00:08:33.159 --> 00:08:36.110
just write this as a system
of two unknowns, which is

00:08:36.110 --> 00:08:37.760
actually probably easier.

00:08:37.760 --> 00:08:38.700
So let's do it that way.

00:08:38.700 --> 00:08:42.200
So this, if I were to write it
as a system of equations, is 6

00:08:42.200 --> 00:08:51.540
times m star plus 2 times
b star, is equal to 4.

00:08:51.540 --> 00:09:00.120
And then I get 2 times m star
plus 4 times b star is

00:09:00.120 --> 00:09:02.830
equal to this 4.

00:09:02.830 --> 00:09:06.950
So let me solve for my m
stars and my b stars.

00:09:06.950 --> 00:09:09.320
So let's multiply this second
equation, actually let's

00:09:09.320 --> 00:09:12.570
multiply that top
equation by 2.

00:09:12.570 --> 00:09:14.280
This is just straight
Algebra 1.

00:09:14.280 --> 00:09:16.300
So times 2, what do we get?

00:09:16.300 --> 00:09:23.970
We get 12m star plus 4b
star is equal to 8.

00:09:23.970 --> 00:09:26.560
We just multiplied that
top guy by 2.

00:09:26.560 --> 00:09:35.930
Now let's multiply this magenta
1 by negative 1.

00:09:35.930 --> 00:09:40.200
So this becomes a minus, this
becomes a minus, that becomes

00:09:40.200 --> 00:09:43.370
a minus, and now we can add
these two equations.

00:09:43.370 --> 00:09:49.490
So we get minus 2 plus 12m
star, that's 10m star.

00:09:49.490 --> 00:09:54.700
And then the minus 4b and the 4b
cancel out, is equal to 4,

00:09:54.700 --> 00:10:01.000
or m star is equal to 4 over
10, which is equal to 2/5.

00:10:01.000 --> 00:10:03.320
Now we can just go and
back-substitute into this.

00:10:03.320 --> 00:10:06.450
We can say 6 times m
star-- This is just

00:10:06.450 --> 00:10:07.990
straight Algebra 1.

00:10:07.990 --> 00:10:13.430
So 6 times our m star, so 6
times 2 over 5, plus 2 times

00:10:13.430 --> 00:10:17.410
our b star is equal to 4.

00:10:17.410 --> 00:10:19.170
Enough white, let
me use yellow.

00:10:19.170 --> 00:10:27.760
So we get 12 over 5 plus 2b
star is equal to 4, or we

00:10:27.760 --> 00:10:33.050
could say 2b star-- let me
scroll down a little bit --2b

00:10:33.050 --> 00:10:35.580
star is equal to 4.

00:10:35.580 --> 00:10:41.350
Which is the same thing as 20
over 5, minus 12 over 5, which

00:10:41.350 --> 00:10:43.970
is equal to-- I'm just
subtracting the 12 over 5 from

00:10:43.970 --> 00:10:47.180
both sides --which is
equal to 8 over 5.

00:10:47.180 --> 00:10:50.160
And you divide both sides of the
equation by 2, you get b

00:10:50.160 --> 00:10:54.220
star is equal to 4/5.

00:10:54.220 --> 00:10:57.940
And just like that, we got our
m star and our b star.

00:10:57.940 --> 00:11:07.770
Our least squares solution
is equal to 2/5 and 4/5.

00:11:07.770 --> 00:11:10.900
So m is equal to 2/5 and
b is equal to 4/5.

00:11:10.900 --> 00:11:13.030
And remember, the whole point
of this was to find an

00:11:13.030 --> 00:11:14.810
equation of the line.

00:11:14.810 --> 00:11:17.890
y is equal to mx plus b.

00:11:17.890 --> 00:11:20.500
Now we can't find a line that
went through all of those

00:11:20.500 --> 00:11:24.240
points up there, but this
is going to be our

00:11:24.240 --> 00:11:25.610
least squares solution.

00:11:25.610 --> 00:11:33.580
This is the one that minimizes
the distance between a times

00:11:33.580 --> 00:11:36.490
our vector and b.

00:11:36.490 --> 00:11:40.580
No vector, when you multiply
times that matrix a-- that's

00:11:40.580 --> 00:11:45.610
not a, that's transpose a --no
other solution is going to

00:11:45.610 --> 00:11:48.880
give us a closer solution to
b than when we put our

00:11:48.880 --> 00:11:51.600
newly-found x star into
this equation.

00:11:51.600 --> 00:11:53.660
This is going to give us
our best solution.

00:11:53.660 --> 00:11:56.800
It's going to minimize
the distance to b.

00:11:56.800 --> 00:11:58.880
So let's write it out.

00:11:58.880 --> 00:12:00.640
y is equal to mx plus b.

00:12:00.640 --> 00:12:05.870
So y is equal to
2/5 x plus 2/5.

00:12:05.870 --> 00:12:07.590
Let's graph that out.

00:12:07.590 --> 00:12:09.650
y is equal to 2/5 x plus 2/5.

00:12:15.890 --> 00:12:20.130
So its y-intercept is 2/5,
which is about there .

00:12:20.130 --> 00:12:21.200
This is at 1.

00:12:21.200 --> 00:12:23.320
2/5 is right about there.

00:12:23.320 --> 00:12:25.020
And then its slope is 2/5.

00:12:27.580 --> 00:12:31.110
Let's think of it this way: for
every 2 and 1/2 you go to

00:12:31.110 --> 00:12:32.510
the right, you're going
to go up 1.

00:12:32.510 --> 00:12:39.170
So if you go 1, 2 and 1/2,
we're going to go up 1.

00:12:39.170 --> 00:12:40.590
We're going to go
up 1 like that.

00:12:40.590 --> 00:12:43.920
So our line-- and obviously this
isn't precise --but our

00:12:43.920 --> 00:12:45.720
line is going to look
something like this.

00:12:54.420 --> 00:12:56.520
I want to do my best shot
at drawing it because

00:12:56.520 --> 00:12:58.260
this is the fun part.

00:12:58.260 --> 00:13:01.820
It's going to look something
like that.

00:13:01.820 --> 00:13:05.440
And that right there is my least
squares estimate for a

00:13:05.440 --> 00:13:07.260
line that goes through
all of those points.

00:13:07.260 --> 00:13:12.360
And you're not going to find a
line that minimizes the error

00:13:12.360 --> 00:13:14.690
in a better way, at least when
you measure the error as the

00:13:14.690 --> 00:13:20.150
distance between this vector
and the vector a times our

00:13:20.150 --> 00:13:22.180
least squares estimate.

00:13:22.180 --> 00:13:24.740
Anyway, thought you would
find that neat.