Quadratic approximation example

WEBVTT
Kind: captions
Language: en

00:00:01.244 --> 00:00:03.625
- [Voiceover] When we last
left off in the riveting saga

00:00:03.625 --> 00:00:06.811
of quadratic approximations
of multi variable functions,

00:00:06.811 --> 00:00:10.298
we were approximating a two
variable function, f of xy,

00:00:10.298 --> 00:00:13.708
and we ended up with this
pretty monstrous expression,

00:00:13.708 --> 00:00:16.140
and because it's written its
full abstract form, I almost

00:00:16.140 --> 00:00:18.651
feel like it looks more
monstrous than it needs to,

00:00:18.651 --> 00:00:20.435
so I'm going to go ahead
and go through a specific

00:00:20.435 --> 00:00:23.150
example here, and just to
remind you of kind of what

00:00:23.150 --> 00:00:24.860
all these terms are, how
there's actually kind of a

00:00:24.860 --> 00:00:27.614
pattern to what's going
on, this here represents,

00:00:27.614 --> 00:00:30.318
you can think of it as the
constant term, where you know

00:00:30.318 --> 00:00:32.687
this is just going to evaluate
to some kind of number.

00:00:32.687 --> 00:00:37.043
These two terms are what you
might call the linear term,

00:00:37.043 --> 00:00:40.288
linear, because if you
actually look, the only places

00:00:40.288 --> 00:00:43.339
where the variable x
and y comes up is here,

00:00:43.339 --> 00:00:45.033
where it's just being
multiplied by a constant,

00:00:45.033 --> 00:00:47.288
and here, where it's just
being multiplied by a constant,

00:00:47.288 --> 00:00:49.638
so it's just variables
times constant in there.

00:00:49.638 --> 00:00:51.714
And then all of this stuff
at the end, which is kind of

00:00:51.714 --> 00:00:54.116
the whole essence of a
quadratic approximation,

00:00:54.116 --> 00:00:56.716
where you start to have things
like you get an x squared,

00:00:56.716 --> 00:00:59.415
and you get x gets to be multiplied by y,

00:00:59.415 --> 00:01:01.958
all of this stuff is the
quadratic term, and though

00:01:01.958 --> 00:01:03.879
it seems like a lot now,
you'll see in the context

00:01:03.879 --> 00:01:06.712
of an actual example, it's
not necessarily as bad

00:01:06.712 --> 00:01:07.799
as it seems.

00:01:07.799 --> 00:01:11.966
So let's say we're looking
at the function f of xy

00:01:14.117 --> 00:01:16.122
and let's say it's going to be

00:01:16.122 --> 00:01:18.534
e to the x divided by two,

00:01:18.534 --> 00:01:20.540
multiplied by sin of y.

00:01:20.540 --> 00:01:23.044
This is our multi variable function.

00:01:23.044 --> 00:01:24.844
And let's say we want to
approximate this near some

00:01:24.844 --> 00:01:26.965
kind of point, and I'm going
to choose a point that's

00:01:26.965 --> 00:01:30.011
something that we can
actually evaluate these at,

00:01:30.011 --> 00:01:33.067
so like x, it would be
convenient if that was zero,

00:01:33.067 --> 00:01:36.468
and then y, I'll go with
pi halves, because that's

00:01:36.468 --> 00:01:38.719
something where I'll
know how to evaluate sin,

00:01:38.719 --> 00:01:41.127
and where I'll know how to
evaluate its derivatives,

00:01:41.127 --> 00:01:42.074
things like that.

00:01:42.074 --> 00:01:44.417
So we're trying to
approximate this function near

00:01:44.417 --> 00:01:45.861
this point.

00:01:45.861 --> 00:01:47.015
Now first things first.

00:01:47.015 --> 00:01:49.116
We're just going to need
to get all of the different

00:01:49.116 --> 00:01:51.057
partial derivatives and
second partial derivatives.

00:01:51.057 --> 00:01:52.933
We know we're going to need
them, so let's just kind

00:01:52.933 --> 00:01:55.246
of start working it through,
and figuring out what

00:01:55.246 --> 00:01:56.579
all of them are.

00:01:57.585 --> 00:02:01.895
Let's start with the partial
derivative with respect to x.

00:02:01.895 --> 00:02:05.428
So this is also a function of xy,

00:02:05.428 --> 00:02:07.194
and we look up at the original function,

00:02:07.194 --> 00:02:09.663
the only place where x shows
up is in this e to the x

00:02:09.663 --> 00:02:12.538
over two, the derivative
of that is one half,

00:02:12.538 --> 00:02:14.122
we bring down that one half,

00:02:14.122 --> 00:02:15.801
times e to the x over two,

00:02:15.801 --> 00:02:17.713
and this is being multiplied
by something that looks

00:02:17.713 --> 00:02:20.338
like a constant, as far as x is concerned,

00:02:20.338 --> 00:02:21.171
sin of y.

00:02:22.259 --> 00:02:26.615
Now when we do the partial
derivative with respect to y,

00:02:26.615 --> 00:02:27.615
what we get,

00:02:28.656 --> 00:02:30.368
this first part just looks
like a constant, so we kind

00:02:30.368 --> 00:02:33.500
of keep that constant there,
as far as y is concerned,

00:02:33.500 --> 00:02:36.498
and the derivative of sin is cosin.

00:02:36.498 --> 00:02:38.936
Cosin of y.

00:02:38.936 --> 00:02:40.632
And then now we

00:02:40.632 --> 00:02:42.546
let's start taking second
partial derivatives,

00:02:42.546 --> 00:02:44.071
so I'll start by doing the one where

00:02:44.071 --> 00:02:48.167
we take the partial derivative
with respect to x twice.

00:02:48.167 --> 00:02:51.032
Now here I'll actually do
this in a different color.

00:02:51.032 --> 00:02:52.759
Let's do yellow.

00:02:52.759 --> 00:02:53.857
Just to make clear which
ones are the second

00:02:53.857 --> 00:02:55.838
partial derivatives.

00:02:55.838 --> 00:02:57.879
So partial with respect to x twice,

00:02:57.879 --> 00:03:01.237
also, function of xy,
like all of these guys.

00:03:01.237 --> 00:03:03.513
So let's look up at the
original partial derivative with

00:03:03.513 --> 00:03:05.661
respect to x, and we're
now going to take its

00:03:05.661 --> 00:03:07.682
derivative, again with respect to x.

00:03:07.682 --> 00:03:09.301
This is the only place where x shows up,

00:03:09.301 --> 00:03:11.481
that one half kind of comes down again,

00:03:11.481 --> 00:03:13.709
so now it's going to be one fourth times

00:03:13.709 --> 00:03:15.716
e to the x over two,

00:03:15.716 --> 00:03:17.214
and we just keep that sin of y,

00:03:17.214 --> 00:03:18.860
because it looks like
we're just multiplying

00:03:18.860 --> 00:03:20.860
by a constant, sin of y.

00:03:22.192 --> 00:03:25.393
Next we'll do the mixed partial
derivative, where you do

00:03:25.393 --> 00:03:27.877
first with respect to x,
then with respect to y,

00:03:27.877 --> 00:03:29.379
or you could do it the other
way, because with almost

00:03:29.379 --> 00:03:30.835
all functions, it kind
of doesn't matter which

00:03:30.835 --> 00:03:32.367
order you take the two.

00:03:32.367 --> 00:03:33.491
So I'll go ahead and
just look at the one that

00:03:33.491 --> 00:03:35.673
was with respect to x,
and now let's think of its

00:03:35.673 --> 00:03:37.598
derivative with respect to y.

00:03:37.598 --> 00:03:39.604
This whole one half e to the x halves

00:03:39.604 --> 00:03:41.776
looks like a constant,
derivative of sin of y

00:03:41.776 --> 00:03:43.109
is cosin y.

00:03:43.109 --> 00:03:47.251
So we take that constant,
the one half e to the

00:03:47.251 --> 00:03:48.418
x halves,

00:03:51.158 --> 00:03:53.190
and then we multiply it by
the derivative of sin of y,

00:03:53.190 --> 00:03:54.857
which is cosin of y.

00:03:56.971 --> 00:03:59.561
And then finally we take
the second derivative,

00:03:59.561 --> 00:04:01.040
second partial derivative

00:04:01.040 --> 00:04:03.873
with respect to y, twice in a row.

00:04:05.320 --> 00:04:07.954
So f with respect to y,

00:04:07.954 --> 00:04:09.204
twice in a row.

00:04:10.855 --> 00:04:12.458
And for this one, let's
take a look at the partial

00:04:12.458 --> 00:04:14.084
derivative with respect to y.

00:04:14.084 --> 00:04:16.384
This part is the only
part where y shows up,

00:04:16.384 --> 00:04:18.579
derivative of cosin is negative sin,

00:04:18.579 --> 00:04:21.330
and then e to the x halves just
still looks like a constant,

00:04:21.330 --> 00:04:23.675
so we'll bring that negative out front,

00:04:23.675 --> 00:04:25.721
that constant e to the x halves,

00:04:25.721 --> 00:04:27.721
and it was negative sin,

00:04:28.830 --> 00:04:30.353
so that negative went out front,

00:04:30.353 --> 00:04:31.896
sin of y.

00:04:31.896 --> 00:04:34.468
So that's all of the partial
differential information

00:04:34.468 --> 00:04:36.211
that we're going to need.

00:04:36.211 --> 00:04:38.388
And now we know we're
going to need to evaluate

00:04:38.388 --> 00:04:41.503
all of these guys, all of
these partial derivatives

00:04:41.503 --> 00:04:43.675
at the specific point,
because if we go up and

00:04:43.675 --> 00:04:46.445
look at the original
function that we have,

00:04:46.445 --> 00:04:48.428
we're going to need to
evaluate f at this point,

00:04:48.428 --> 00:04:50.431
both of the partial
derivatives at this point,

00:04:50.431 --> 00:04:53.414
the second partial derivatives.

00:04:53.414 --> 00:04:54.969
Oh, I'm realizing, actually,
that I made a little

00:04:54.969 --> 00:04:56.079
bit of a mistake here.

00:04:56.079 --> 00:04:58.206
This should be a one half

00:04:58.206 --> 00:04:59.820
out in front of each of these guys.

00:04:59.820 --> 00:05:03.303
That should be plus
one half of this second

00:05:03.303 --> 00:05:06.042
partial derivative, and
one half of this second

00:05:06.042 --> 00:05:07.221
partial derivative.

00:05:07.221 --> 00:05:08.727
The mixed partial
derivative it's still one,

00:05:08.727 --> 00:05:10.010
but these guys should have a one half.

00:05:10.010 --> 00:05:12.311
That was a mistake on my part.

00:05:12.311 --> 00:05:14.075
In any case, though, we're
going to need to evaluate

00:05:14.075 --> 00:05:16.575
all of these guys, so if we go back down,

00:05:16.575 --> 00:05:19.132
let's just start plugging
in the point zero and

00:05:19.132 --> 00:05:21.146
pi halves to each one of these.

00:05:21.146 --> 00:05:23.324
So the function itself,
when we plug in zero,

00:05:23.324 --> 00:05:26.459
e to the zero is one,
and sin of pi halves,

00:05:26.459 --> 00:05:28.339
sin of pi halves is also one,

00:05:28.339 --> 00:05:31.452
so this entire thing just comes to one.

00:05:31.452 --> 00:05:33.562
If we do this for the next one, again

00:05:33.562 --> 00:05:36.481
e to the zero is going to be one,

00:05:36.481 --> 00:05:37.978
sin of y is also going to be one,

00:05:37.978 --> 00:05:40.045
but now we have that
one half sitting there,

00:05:40.045 --> 00:05:42.293
so that'll end up as one half.

00:05:42.293 --> 00:05:44.014
If we look at the partial
derivative with respect

00:05:44.014 --> 00:05:46.764
to y, cosin of pi halves is zero,

00:05:47.686 --> 00:05:51.169
so this entire thing is going to be zero.

00:05:51.169 --> 00:05:53.640
Moving right along, let's
take a look at this second

00:05:53.640 --> 00:05:56.024
partial derivative with respect to x.

00:05:56.024 --> 00:05:58.359
Again, e to the zero will be one,

00:05:58.359 --> 00:06:00.700
and sin of pi halves will be one,

00:06:00.700 --> 00:06:04.832
so this ends up just
being that one fourth,

00:06:04.832 --> 00:06:07.108
the mixed partial derivative here,

00:06:07.108 --> 00:06:10.788
if we have one half by the
pattern's starting to continue,

00:06:10.788 --> 00:06:12.987
you've got the one, this
one's actually zero,

00:06:12.987 --> 00:06:15.640
so cosin of pi halves is zero,

00:06:15.640 --> 00:06:17.317
so the whole thing will be zero.

00:06:17.317 --> 00:06:20.098
And then the last one it'll be negative

00:06:20.098 --> 00:06:22.259
one times that one again,

00:06:22.259 --> 00:06:24.654
for sin of pi halves is one,

00:06:24.654 --> 00:06:28.573
so all of that just comes
out to be negative one.

00:06:28.573 --> 00:06:30.999
So this, I mean I kind of
chose a convenient example,

00:06:30.999 --> 00:06:33.006
where all the derivatives
look very similar to the

00:06:33.006 --> 00:06:35.503
thing itself, which is
actually pretty common,

00:06:35.503 --> 00:06:37.912
so we get to leverage
a lot of the work that

00:06:37.912 --> 00:06:39.222
we did earlier.

00:06:39.222 --> 00:06:41.270
So now we have these
six different constants,

00:06:41.270 --> 00:06:44.320
can't keep them all on the
screen at the same time.

00:06:44.320 --> 00:06:46.781
But we've got these six
different constants,

00:06:46.781 --> 00:06:48.683
so now we just plug each
one of them in to the

00:06:48.683 --> 00:06:51.094
quadratic approximation.

00:06:51.094 --> 00:06:55.495
So if we make our quadratic
approximation of our function,

00:06:55.495 --> 00:06:57.295
the first term is that constant term,

00:06:57.295 --> 00:07:00.412
so we take a look up and
we say where does f of xy

00:07:00.412 --> 00:07:03.942
go at this point, and it'll just be one.

00:07:03.942 --> 00:07:05.077
We're going to have to do
a lot of scrolling back

00:07:05.077 --> 00:07:06.016
and forth here.

00:07:06.016 --> 00:07:08.065
There's a lot of text to deal with.

00:07:08.065 --> 00:07:09.830
The next thing is going to be something

00:07:09.830 --> 00:07:14.405
times x minus zero, the
kind of x coordinate

00:07:14.405 --> 00:07:15.892
of our specified point,

00:07:15.892 --> 00:07:17.785
and that something is
the first derivative with

00:07:17.785 --> 00:07:20.713
respect to x, so that's
going to be one half.

00:07:20.713 --> 00:07:22.530
So come back down here.

00:07:22.530 --> 00:07:24.518
That one half.

00:07:24.518 --> 00:07:26.111
And then similarly, we're
going to have something

00:07:26.111 --> 00:07:28.988
multiplied by y minus the y coordinate

00:07:28.988 --> 00:07:32.380
of the point about which
we are approximating,

00:07:32.380 --> 00:07:34.790
and for that we take a look
at the partial derivative

00:07:34.790 --> 00:07:36.589
with respect to y, which was just zero,

00:07:36.589 --> 00:07:38.210
so that's pretty convenient.

00:07:38.210 --> 00:07:40.524
That's just going to end up being zero.

00:07:40.524 --> 00:07:42.538
And then for the second
partial derivative terms,

00:07:42.538 --> 00:07:43.953
maybe I'll actually be able to keep it on

00:07:43.953 --> 00:07:45.631
the same screen here.

00:07:45.631 --> 00:07:48.635
We're going to have
something multiplied by x

00:07:48.635 --> 00:07:51.786
minus its coordinate squared,

00:07:51.786 --> 00:07:54.053
and that something is whatever
the partial derivative

00:07:54.053 --> 00:07:56.018
with respect to x twice
is, which is one fourth,

00:07:56.018 --> 00:07:59.185
so we go ahead and plug in one fourth,

00:08:00.203 --> 00:08:01.841
and then for the mixed partial derivative,

00:08:01.841 --> 00:08:03.082
I'll put it down here,

00:08:03.082 --> 00:08:07.565
it'll be something multiplied
by x, minus its constant,

00:08:07.565 --> 00:08:08.898
and then y minus

00:08:10.459 --> 00:08:11.709
that pi halves,

00:08:12.748 --> 00:08:14.937
and that something is the
mixed partial derivative,

00:08:14.937 --> 00:08:16.803
which in this case is zero.

00:08:16.803 --> 00:08:18.482
Oh and I'm realizing I made
the same mistake again.

00:08:18.482 --> 00:08:21.399
It's not one fourth, it's one half.

00:08:22.416 --> 00:08:24.690
For the same reason that I
made a mistake up here earlier,

00:08:24.690 --> 00:08:26.456
where it's actually one
half multiplied by this

00:08:26.456 --> 00:08:28.956
second partial derivative,
and one half by the second

00:08:28.956 --> 00:08:32.189
partial derivative there, I
guess I keep forgetting that.

00:08:32.189 --> 00:08:35.434
Good lesson, I suppose, that
that's an easy thing to forget,

00:08:35.434 --> 00:08:37.673
if you find yourself
computing one of these,

00:08:37.673 --> 00:08:39.714
where I'll put it in here,

00:08:39.714 --> 00:08:41.677
multiply that guy by one half.

00:08:41.677 --> 00:08:44.226
It's similar to a Taylor
expansion in single variable

00:08:44.226 --> 00:08:45.750
calculus, where you kind
of have to remember what's

00:08:45.750 --> 00:08:48.323
what that squared term
would be, has a one half

00:08:48.323 --> 00:08:50.563
associated with it.

00:08:50.563 --> 00:08:52.284
So for that same reason,
now we're going to have,

00:08:52.284 --> 00:08:54.773
and this time I won't forget it,

00:08:54.773 --> 00:08:57.683
will be one half multiplied by something,

00:08:57.683 --> 00:09:00.683
multiplied by the y minus pi halves,

00:09:02.008 --> 00:09:04.648
minus that y coordinate of
the point we're approximating

00:09:04.648 --> 00:09:06.490
there, and this time that something

00:09:06.490 --> 00:09:08.573
is negative one.

00:09:09.955 --> 00:09:13.764
So we can kind of plug
in here negative one.

00:09:13.764 --> 00:09:16.783
And now this is something
we can simplify quite a bit,

00:09:16.783 --> 00:09:19.200
because that one stays there,

00:09:20.141 --> 00:09:21.759
one half of x minus zero,

00:09:21.759 --> 00:09:23.853
that's just x halves,

00:09:23.853 --> 00:09:25.324
this whole part cancels out to zero,

00:09:25.324 --> 00:09:26.692
so there's nothing there.

00:09:26.692 --> 00:09:29.103
Over here we have half
times a fourth, one eighth

00:09:29.103 --> 00:09:31.507
times x squared, so that's

00:09:31.507 --> 00:09:33.757
x squared divided by eight.

00:09:34.908 --> 00:09:36.812
This mixed partial
derivative term is zero,

00:09:36.812 --> 00:09:38.118
so that's pretty nice.

00:09:38.118 --> 00:09:40.865
And then this last term here is just

00:09:40.865 --> 00:09:42.957
negative one half,

00:09:42.957 --> 00:09:46.684
so let's see, I'll write
it down as negative

00:09:46.684 --> 00:09:50.581
one half by y minus pi halves squared,

00:09:50.581 --> 00:09:53.831
by y minus pi halves squared.

00:09:56.241 --> 00:09:58.764
So that is the quadratic approximation,

00:09:58.764 --> 00:10:00.563
and you can see this actually feels like a

00:10:00.563 --> 00:10:01.562
quadratic function.

00:10:01.562 --> 00:10:05.230
We've got up to x squared,
and up to y squared,

00:10:05.230 --> 00:10:07.700
and there's a sense in which this is a

00:10:07.700 --> 00:10:08.907
simpler function.

00:10:08.907 --> 00:10:10.612
I mean, it looks like it's
got more terms than the

00:10:10.612 --> 00:10:13.788
original one, which was e
to the x halved sin of y,

00:10:13.788 --> 00:10:16.138
but if it's a computer
that needs to compute these

00:10:16.138 --> 00:10:18.885
things, for example, it's
much easier to deal with

00:10:18.885 --> 00:10:21.339
polynomials, that's a faster thing to do.

00:10:21.339 --> 00:10:25.131
Also for theoretical
purposes, it can be nice to

00:10:25.131 --> 00:10:27.099
deal with just a quadratic
polynomial to make

00:10:27.099 --> 00:10:28.639
conclusions about things.

00:10:28.639 --> 00:10:30.209
We'll see that in the context
of something called the

00:10:30.209 --> 00:10:32.222
second partial derivative test.

00:10:32.222 --> 00:10:34.796
But just to get a feel for
what this means, let's pull

00:10:34.796 --> 00:10:38.046
up the graph of the relevant functions.

00:10:39.628 --> 00:10:41.555
So this here is the graph
of the original function,

00:10:41.555 --> 00:10:43.958
e to the x halves times sin of y,

00:10:43.958 --> 00:10:46.394
and the point that
we're approximating near

00:10:46.394 --> 00:10:48.356
was where x equals zero,

00:10:48.356 --> 00:10:50.147
so let's see how we get oriented.

00:10:50.147 --> 00:10:52.652
X is equal to zero, and then
y is equal to pi halves.

00:10:52.652 --> 00:10:54.772
So this is the point
we're approximating near.

00:10:54.772 --> 00:10:56.786
And the quadratic approximation,
when you plug everything

00:10:56.786 --> 00:11:00.178
in, has a graph that looks
like this white surface here.

00:11:00.178 --> 00:11:02.522
So if I get rid of that
original graph, this is how

00:11:02.522 --> 00:11:04.991
we're approximating the
function near that point.

00:11:04.991 --> 00:11:06.630
And that does a pretty good job, right?

00:11:06.630 --> 00:11:08.778
Because even as you step
pretty far away from that

00:11:08.778 --> 00:11:11.909
point, it's pretty closely
hugging the original surface.

00:11:11.909 --> 00:11:13.927
If you go very far away,
it certainly doesn't get

00:11:13.927 --> 00:11:16.364
the oscillating nature
of that sin component,

00:11:16.364 --> 00:11:18.767
and the exponential
component grows faster than

00:11:18.767 --> 00:11:20.807
the quadratic one, but nearby,

00:11:20.807 --> 00:11:23.553
nearby this actually
gives a very good feel for

00:11:23.553 --> 00:11:25.432
the shape of the graph.

00:11:25.432 --> 00:11:26.938
And again, later on we'll
see how this is a pretty

00:11:26.938 --> 00:11:29.805
useful theoretical tool for
drawing conclusions about

00:11:29.805 --> 00:11:31.836
qualitative features of
the shape of the graph,

00:11:31.836 --> 00:11:34.213
the fact that this looks
kind of like a saddle,

00:11:34.213 --> 00:11:35.765
is going to end up being kind of important

00:11:35.765 --> 00:11:37.295
in certain contexts.

00:11:37.295 --> 00:11:39.328
But before we get to any
of that, in the next couple

00:11:39.328 --> 00:11:41.982
videos I'm going to talk about a simpler,

00:11:41.982 --> 00:11:44.398
or rather a more generalizable form,

00:11:44.398 --> 00:11:46.404
of writing down this
quadratic approximation

00:11:46.404 --> 00:11:48.118
using vector notation.

00:11:48.118 --> 00:11:49.720
Because right now we're just limited to

00:11:49.720 --> 00:11:52.070
two variables, and you can
imagine how monstrous this

00:11:52.070 --> 00:11:54.127
might look if you were
dealing even just with a

00:11:54.127 --> 00:11:55.334
three variable function,

00:11:55.334 --> 00:11:57.235
where think of all the
different possible second

00:11:57.235 --> 00:11:59.442
partial derivatives of a
three variable function,

00:11:59.442 --> 00:12:01.471
or a four variable function.

00:12:01.471 --> 00:12:03.572
It would quickly get out
of hand, but there is kind

00:12:03.572 --> 00:12:06.819
of a nice general way
to write all of these.

00:12:06.819 --> 00:12:08.638
So with that, I will see you next video.


The Hessian matrix

WEBVTT
Kind: captions
Language: en

00:00:00.592 --> 00:00:01.425
- [Voiceover] Hey guys.

00:00:01.425 --> 00:00:04.209
Before talking about the
vector form for the quadratic

00:00:04.209 --> 00:00:06.917
approximation of multivariable functions,

00:00:06.917 --> 00:00:11.084
I've got to introduce this
thing called the Hessian matrix.

00:00:14.229 --> 00:00:16.645
Essentially what this is,
is just a way to package

00:00:16.645 --> 00:00:19.498
all the information of the
second derivatives of a function.

00:00:19.498 --> 00:00:23.249
Let's say you have some kind
of multivariable function

00:00:23.249 --> 00:00:25.965
like the example we had in the last video,

00:00:25.965 --> 00:00:29.198
e to the x halves multiplied by sine of y,

00:00:29.198 --> 00:00:31.006
so some kind of of a
multivariable function.

00:00:31.006 --> 00:00:33.101
What the Hessian matrix
is, and it's often denoted

00:00:33.101 --> 00:00:37.101
with an H, but a bold
faced H, is it's a matrix,

00:00:38.862 --> 00:00:41.120
incidentally enough, that contains all

00:00:41.120 --> 00:00:43.358
the second partial derivatives of f.

00:00:43.358 --> 00:00:46.821
The first component is gonna
be, the partial derivative

00:00:46.821 --> 00:00:50.466
of f with respect to x twice
in a row, and everything

00:00:50.466 --> 00:00:52.413
in this first column is
kind of like you first do it

00:00:52.413 --> 00:00:56.294
with respect to x, because
the next part is the second

00:00:56.294 --> 00:00:59.788
derivative where first you
do it with respect to x

00:00:59.788 --> 00:01:03.001
and then you do it with respect to y.

00:01:03.001 --> 00:01:05.134
That's the first column of the matrix.

00:01:05.134 --> 00:01:10.115
Then up here it's the partial
derivative where first you do

00:01:10.115 --> 00:01:13.935
it with respect to y and then
you do it with respect to x,

00:01:13.935 --> 00:01:15.497
and then over here it's
where you do it with

00:01:15.497 --> 00:01:18.193
respect to y both times in a row.

00:01:18.193 --> 00:01:22.026
Partial with respect to
y both times in a row.

00:01:22.966 --> 00:01:24.408
Let's go ahead and actually compute this

00:01:24.408 --> 00:01:25.997
and think about what this would look like

00:01:25.997 --> 00:01:28.363
in the case of our specific function here.

00:01:28.363 --> 00:01:30.609
In order to get all the second
partial derivatives we first

00:01:30.609 --> 00:01:33.869
should keep a record of the
first partial derivatives.

00:01:33.869 --> 00:01:37.697
The partial derivative
of f with respect to x.

00:01:37.697 --> 00:01:41.056
The only place x shows up is
in this e to the x halves.

00:01:41.056 --> 00:01:44.712
Bring down that 1/2 e to
the x halves and sine of y

00:01:44.712 --> 00:01:47.734
just looks like a constant
as far as x is concerned.

00:01:47.734 --> 00:01:48.942
Sine of y.

00:01:48.942 --> 00:01:51.582
Then the partial derivative
with respect of y.

00:01:51.582 --> 00:01:55.362
Partial derivative of f with respect to y.

00:01:55.362 --> 00:01:57.830
Now e to the x halves
looks like a constant

00:01:57.830 --> 00:02:00.626
and it's being multiplied by
something that has a y in it,

00:02:00.626 --> 00:02:02.416
e to the x halves.

00:02:02.416 --> 00:02:04.530
The derivative of sine of
y, since we're doing it

00:02:04.530 --> 00:02:07.280
with respect to y is cosine of y.

00:02:09.184 --> 00:02:11.659
These terms won't be included
in the Hessian itself

00:02:11.659 --> 00:02:14.114
but we're just keeping a record of them

00:02:14.114 --> 00:02:16.309
because now when we go
into fill in the matrix,

00:02:16.309 --> 00:02:19.115
this upper left component,
we're taking the second partial

00:02:19.115 --> 00:02:22.142
derivative where we do it with
respect to x then x again.

00:02:22.142 --> 00:02:25.352
Up here is when we did
it with respect to x,

00:02:25.352 --> 00:02:27.536
if we did it with respect
to x again we bring down

00:02:27.536 --> 00:02:32.291
another 1/2 so that becomes
1/4 by e to the x halves

00:02:32.291 --> 00:02:36.953
and that sine of y just
still looks like a constant.

00:02:36.953 --> 00:02:38.890
Then this mixed partial
derivative where we do it with

00:02:38.890 --> 00:02:42.631
respect to x then y, so we
did it with respect to x here.

00:02:42.631 --> 00:02:44.642
When we differentiate
this with respect to y,

00:02:44.642 --> 00:02:48.809
the 1/2 e to the x halves
just looks like a constant

00:02:49.791 --> 00:02:53.958
but then derivative of sine
of y ends up as cosine of y.

00:02:56.020 --> 00:02:59.459
Then up here, it's gonna be
the same thing but let's see

00:02:59.459 --> 00:03:00.593
when you do it in the other direction,

00:03:00.593 --> 00:03:03.010
when you do it first
with respect to y then x.

00:03:03.010 --> 00:03:05.193
Over here we did it
first with respect to y.

00:03:05.193 --> 00:03:07.603
If we took this derivative
with respect to x,

00:03:07.603 --> 00:03:10.079
you'd have the 1/2 would
come down, so that would be

00:03:10.079 --> 00:03:14.246
1/2 e to the x halves
multiplied by cosine of y

00:03:15.971 --> 00:03:17.667
because that's just looks
like a constant since

00:03:17.667 --> 00:03:19.736
we're doing it with respect
to x the second time.

00:03:19.736 --> 00:03:24.625
That would be cosine of y,
and it shouldn't feel like a

00:03:24.625 --> 00:03:27.537
surprise that both of these
terms turn out to be the same.

00:03:27.537 --> 00:03:29.298
With most functions that's the case.

00:03:29.298 --> 00:03:30.375
Technically not all functions.

00:03:30.375 --> 00:03:32.297
You can come up with some crazy things

00:03:32.297 --> 00:03:34.246
where this won't be
symmetric, where you’ll

00:03:34.246 --> 00:03:36.951
have different terms in the
diagonal, but for the most part

00:03:36.951 --> 00:03:39.580
those you can expect to be the same.

00:03:39.580 --> 00:03:41.458
In this last term here where
we do it with respect to

00:03:41.458 --> 00:03:44.558
y twice, we now think of taking
the derivative of this whole

00:03:44.558 --> 00:03:48.206
term with respect to y, that
e to the x halves looks like

00:03:48.206 --> 00:03:52.373
a constant and derivative of
cosine is negative sine of y.

00:03:56.118 --> 00:03:59.653
This whole thing, a matrix,
each of whose components

00:03:59.653 --> 00:04:01.940
is a multivariable
function, is the Hessian.

00:04:01.940 --> 00:04:05.080
This is the Hessian of f,
and sometimes bold write it

00:04:05.080 --> 00:04:09.633
as Hessian of f specifying
what function its of.

00:04:09.633 --> 00:04:12.410
You could think of it as
a matrix valued function

00:04:12.410 --> 00:04:15.061
which feels kind of weird
but you plug in two different

00:04:15.061 --> 00:04:17.195
values, x and y, and you'll get a matrix,

00:04:17.195 --> 00:04:19.420
so it's this matrix valued function.

00:04:19.420 --> 00:04:20.814
The nice thing about
writing it like this is that

00:04:20.814 --> 00:04:22.631
you can actually extend
this so that rather

00:04:22.631 --> 00:04:24.866
than just for functions
that have two variables,

00:04:24.866 --> 00:04:27.610
let's say you had a function,
kind of like clear this up,

00:04:27.610 --> 00:04:29.675
let's say u had a function
that had three variables

00:04:29.675 --> 00:04:32.263
or four variables or any number.

00:04:32.263 --> 00:04:35.202
Let's say it was a
function of x, y, and z,

00:04:35.202 --> 00:04:38.514
then you can follow this
pattern and following down

00:04:38.514 --> 00:04:40.787
the first column here the
next term that you would get

00:04:40.787 --> 00:04:44.106
would be the second
partial derivative of f,

00:04:44.106 --> 00:04:46.529
where first you do with respect to x,

00:04:46.529 --> 00:04:48.617
and then you do it with respect to z.

00:04:48.617 --> 00:04:51.584
Then over here it would be
the second partial derivative

00:04:51.584 --> 00:04:55.417
of f, where first you
did it with respect to y

00:04:56.960 --> 00:04:59.567
and then you do it with respect to z,

00:04:59.567 --> 00:05:03.157
I'll clear up even more room here,

00:05:03.157 --> 00:05:05.013
because you'd have another
column where you'd have the

00:05:05.013 --> 00:05:08.661
second partial derivative,
where this time everything first

00:05:08.661 --> 00:05:12.054
you do it with respect to z
and then with respect to x.

00:05:12.054 --> 00:05:14.910
Then over here you'd have
the second partial derivative

00:05:14.910 --> 00:05:16.627
where first you do it with respect

00:05:16.627 --> 00:05:19.092
to z and then with respect to y.

00:05:19.092 --> 00:05:20.790
Then there is the very
last component you'd have

00:05:20.790 --> 00:05:23.919
the second partial derivative
where first you do it with

00:05:23.919 --> 00:05:28.014
respect to, well, I guess you
do it with respect to z twice.

00:05:28.014 --> 00:05:31.114
This whole thing, this three
by three matrix would be

00:05:31.114 --> 00:05:33.999
the Hessian of a three variable function.

00:05:33.999 --> 00:05:35.798
You can see how you could
extend this pattern where if it

00:05:35.798 --> 00:05:38.349
was a four variable function
you'd get a four by four matrix

00:05:38.349 --> 00:05:40.830
of all of the possible
second partial derivatives.

00:05:40.830 --> 00:05:42.769
If it was a 100 variable function

00:05:42.769 --> 00:05:45.555
you would have a 100 by 100 matrix.

00:05:45.555 --> 00:05:47.827
The nice thing about having
this is then we can talk

00:05:47.827 --> 00:05:50.158
about that by just referencing the symbol

00:05:50.158 --> 00:05:51.594
and we'll see in the
next video how this makes

00:05:51.594 --> 00:05:54.300
it very nice to express,
for example the quadratic

00:05:54.300 --> 00:05:57.055
approximation of any kind
of multivariable function

00:05:57.055 --> 00:05:59.525
not just a two variable
function and the symbols

00:05:59.525 --> 00:06:01.095
don't get way out of hand
'cause you don't have to

00:06:01.095 --> 00:06:03.446
reference each one of these
individual components.

00:06:03.446 --> 00:06:05.131
You can just reference
the matrix as a whole

00:06:05.131 --> 00:06:07.794
and start doing matrix operations.

00:06:07.794 --> 00:06:09.379
I will see you in that next video.


Vector form of multivariable quadratic approximation

WEBVTT
Kind: captions
Language: en

00:00:00.245 --> 00:00:02.226
- [Voiceover] Okay, so
we are finally ready

00:00:02.226 --> 00:00:04.937
to express the quadratic approximation

00:00:04.937 --> 00:00:07.752
of a multivariable
function in vector form.

00:00:07.752 --> 00:00:09.517
So, I have the whole
thing written out here

00:00:09.517 --> 00:00:12.413
where f is the function that
we are trying to approximate,

00:00:12.413 --> 00:00:14.782
x naught, y naught is the constant point

00:00:14.782 --> 00:00:16.652
about which we are approximating,

00:00:16.652 --> 00:00:18.478
and then this entire expression

00:00:18.478 --> 00:00:20.109
is the quadratic approximation,

00:00:20.109 --> 00:00:21.261
which I've talked about in past videos,

00:00:21.261 --> 00:00:23.378
and if it seems very
complicated or absurd,

00:00:23.378 --> 00:00:24.559
or you're unfamiliar with it,

00:00:24.559 --> 00:00:26.221
I'm just dissecting it real quick.

00:00:26.221 --> 00:00:27.870
This over here is the Constant term,

00:00:27.870 --> 00:00:29.942
this is just gonna evaluate to a constant,

00:00:29.942 --> 00:00:34.007
everything over here is the Linear term,

00:00:34.007 --> 00:00:35.635
because it just involves taking a variable

00:00:35.635 --> 00:00:37.263
multiplied by a constant,

00:00:37.263 --> 00:00:40.774
and then the remainder,
every one of these components

00:00:40.774 --> 00:00:42.903
will have two variables
multiplied into it.

00:00:42.903 --> 00:00:45.472
So x squared comes up, and x times y,

00:00:45.472 --> 00:00:49.639
and y squared comes up, so
that's the quadratic term.

00:00:50.582 --> 00:00:51.415
Quadratic.

00:00:53.343 --> 00:00:54.897
Now, to vectorize things, first of all,

00:00:54.897 --> 00:00:57.878
let's write down the input,
the input variable (x,y)

00:00:57.878 --> 00:00:59.815
as a vector, and typically we'll do that

00:00:59.815 --> 00:01:03.094
with a boldfaced x to
indicate that it's a vector,

00:01:03.094 --> 00:01:07.088
and its components are just
gonna be the single variables,

00:01:07.088 --> 00:01:08.928
x and y, the non-boldfaced.

00:01:08.928 --> 00:01:11.750
So this is the vector
representing the variable input,

00:01:11.750 --> 00:01:14.543
and then correspondingly a boldfaced x

00:01:14.543 --> 00:01:17.031
with a little subscript o, x naught,

00:01:17.031 --> 00:01:19.320
is gonna be the constant input,

00:01:19.320 --> 00:01:23.487
the single point in space near
which we are approximating.

00:01:24.471 --> 00:01:25.871
So when we write things like that,

00:01:25.871 --> 00:01:27.455
this Constant term, simply enough,

00:01:27.455 --> 00:01:29.493
is gonna look like
evaluating your function

00:01:29.493 --> 00:01:31.993
at that boldfaced x naught.

00:01:32.987 --> 00:01:35.070
So that's probably the
easiest one to handle.

00:01:35.070 --> 00:01:38.075
Now the linear term, this
looks like a dot product,

00:01:38.075 --> 00:01:40.757
and if we kind of expand
it out as the dot product,

00:01:40.757 --> 00:01:43.530
it looks like we're taking
the partial derivative

00:01:43.530 --> 00:01:44.957
of f with respect to x,

00:01:44.957 --> 00:01:47.504
and then the partial
derivative with respect to y,

00:01:47.504 --> 00:01:49.325
and we're evaluating both of those

00:01:49.325 --> 00:01:52.325
at that boldfaced x naught input.

00:01:53.652 --> 00:01:55.844
X naught as its input.

00:01:55.844 --> 00:01:57.497
Now, each one of those partial derivatives

00:01:57.497 --> 00:01:59.610
is multiplied by variable
minus constant numbers,

00:01:59.610 --> 00:02:02.049
so this looks like taking the dot product,

00:02:02.049 --> 00:02:05.382
here, I'm gonna erase the word "linear".

00:02:07.683 --> 00:02:10.927
We're taking with x minus x naught,

00:02:10.927 --> 00:02:12.677
and y minus y naught.

00:02:14.354 --> 00:02:16.307
This is just expressing
the same linear term,

00:02:16.307 --> 00:02:18.537
but as a dot product,
but the convenience here

00:02:18.537 --> 00:02:20.449
is that this is totally the same thing

00:02:20.449 --> 00:02:22.782
as saying the gradient of f,

00:02:23.851 --> 00:02:26.392
that's the vector that contains
all the partial derivatives,

00:02:26.392 --> 00:02:30.059
evaluated at the special input, x naught,

00:02:31.402 --> 00:02:33.800
and then we're taking the
dot product between that

00:02:33.800 --> 00:02:37.967
and the variable vector,
boldfaced x, minus x naught.

00:02:40.747 --> 00:02:42.181
Since when you do this component-wise,

00:02:42.181 --> 00:02:44.293
boldfaced x minus x naught,
if we kinda think here,

00:02:44.293 --> 00:02:46.510
it'll be x the variable
minus x naught the constant,

00:02:46.510 --> 00:02:48.583
y the variable minus
y naught the constant,

00:02:48.583 --> 00:02:50.292
which is what we have up there.

00:02:50.292 --> 00:02:53.045
So this expression kind of
vectorizes the whole linear term,

00:02:53.045 --> 00:02:54.966
and now the beef here, the hard part,

00:02:54.966 --> 00:02:58.055
how are we gonna vectorize
this quadratic term?

00:02:58.055 --> 00:03:00.324
Now that's what I was leading
to in the last couple videos,

00:03:00.324 --> 00:03:03.071
where I talked about how
you express a quadratic form

00:03:03.071 --> 00:03:06.732
like this with a matrix,
and the way that you do it,

00:03:06.732 --> 00:03:08.416
I'm just kinda scroll
down to give us some room,

00:03:08.416 --> 00:03:11.748
the way that you do it
is we'll have a matrix

00:03:11.748 --> 00:03:13.848
whose components are
all of these constants.

00:03:13.848 --> 00:03:16.290
It'll be this 1/2 times the
second partial derivative

00:03:16.290 --> 00:03:18.828
evaluated there, and I'm just
gonna, for convenience's sake,

00:03:18.828 --> 00:03:21.757
I'm gonna just take 1/2 times
the second partial derivative

00:03:21.757 --> 00:03:23.486
with respect to x, and
leave it as understood

00:03:23.486 --> 00:03:25.524
that we're evaluating it at this point.

00:03:25.524 --> 00:03:27.373
And then, on the other diagonal,

00:03:27.373 --> 00:03:30.460
you have 1/2 times the other
kind of partial derivative

00:03:30.460 --> 00:03:33.791
with respect to y two times in a row.

00:03:33.791 --> 00:03:36.782
And then we're gonna multiply
it by this constant here,

00:03:36.782 --> 00:03:38.249
but this term kind of gets broken apart

00:03:38.249 --> 00:03:39.589
into two different components.

00:03:39.589 --> 00:03:41.276
If you'll remember, in
the quadratic form video,

00:03:41.276 --> 00:03:43.015
it was always things where it was a,

00:03:43.015 --> 00:03:47.350
and then 2b and c, as your
constants for the quadratic form,

00:03:47.350 --> 00:03:49.794
so if we're interpreting
this as two times something,

00:03:49.794 --> 00:03:51.982
then it gets broken down,

00:03:51.982 --> 00:03:54.815
and on one corner shows up as fxy,

00:03:56.190 --> 00:04:00.088
and on the other one, kind of 1/2 fxy.

00:04:00.088 --> 00:04:01.832
So both of these together
are gonna constitute

00:04:01.832 --> 00:04:04.832
the entire mixed partial derivative.

00:04:06.192 --> 00:04:08.599
And then the way that we
express the quadratic form

00:04:08.599 --> 00:04:11.824
is we're gonna multiply
this by, well by what?

00:04:11.824 --> 00:04:14.056
Well, the first component
is whatever the thing is

00:04:14.056 --> 00:04:18.813
that's squared here, so it's
gonna be that x minus x naught,

00:04:18.813 --> 00:04:20.024
and then the second component is

00:04:20.024 --> 00:04:22.151
whatever the other thing squared is,

00:04:22.151 --> 00:04:25.401
which in this case is y minus y naught,

00:04:28.540 --> 00:04:29.863
and of course we take that same vector

00:04:29.863 --> 00:04:31.391
but we put it in on the other side too.

00:04:31.391 --> 00:04:34.192
So let me make a little bit of room,

00:04:34.192 --> 00:04:36.569
cause this is gonna be wide.

00:04:36.569 --> 00:04:39.019
So we're gonna take that same vector,

00:04:39.019 --> 00:04:40.740
and then kind of put it on its side.

00:04:40.740 --> 00:04:44.594
So it'll be x minus x naught
as the first component,

00:04:44.594 --> 00:04:47.865
and then y minus y naught
as the second component,

00:04:47.865 --> 00:04:50.329
but it's written horizontally, and this,

00:04:50.329 --> 00:04:52.332
if you multiply out the entire matrix,

00:04:52.332 --> 00:04:54.875
is gonna give us the same
expression that you have up here.

00:04:54.875 --> 00:04:56.658
And if that seems
unfamiliar, if that seems,

00:04:56.658 --> 00:04:58.556
you know, how do you
go from there to there,

00:04:58.556 --> 00:05:00.749
check out the video on quadratic forms,

00:05:00.749 --> 00:05:02.886
or you can check out the
article where I'm talking

00:05:02.886 --> 00:05:05.271
about the quadratic
approximation as a whole,

00:05:05.271 --> 00:05:07.391
I kind of go through
the computation there.

00:05:07.391 --> 00:05:10.958
Now, this matrix right here
is almost the Hessian matrix,

00:05:10.958 --> 00:05:13.174
this is why I made a video
about the Hessian matrix.

00:05:13.174 --> 00:05:14.485
It's not quite, because everything

00:05:14.485 --> 00:05:16.101
has a 1/2 multiplied into it,

00:05:16.101 --> 00:05:17.229
so I'm just gonna kinda take that out

00:05:17.229 --> 00:05:18.612
and we'll remember we have to multiply

00:05:18.612 --> 00:05:20.693
a 1/2 in at some point, but otherwise,

00:05:20.693 --> 00:05:22.949
it is the Hessian matrix, which we denote

00:05:22.949 --> 00:05:26.274
with a kind of boldfaced H, boldfaced H,

00:05:26.274 --> 00:05:28.799
and emphasize that it's the Hessian of f.

00:05:28.799 --> 00:05:30.813
The Hessian is something
you take of a function.

00:05:30.813 --> 00:05:32.493
And like I said, remember
each of these terms

00:05:32.493 --> 00:05:34.352
we should be thinking of as evaluated

00:05:34.352 --> 00:05:36.068
on the special input point,

00:05:36.068 --> 00:05:38.990
evaluating it at that special, you know,

00:05:38.990 --> 00:05:40.812
boldfaced x naught input point.

00:05:40.812 --> 00:05:42.515
I was just kind of too
lazy to write it in,

00:05:42.515 --> 00:05:44.064
each time the x naught y
naught, x naught y naught,

00:05:44.064 --> 00:05:45.729
x naught y naught, all of that.

00:05:45.729 --> 00:05:48.896
But what we have then is we're
multiplying it on the right

00:05:48.896 --> 00:05:53.543
by this whole vector is the
variable vector, boldfaced x,

00:05:53.543 --> 00:05:58.456
minus boldfaced x naught, that's
what that entire vector is,

00:05:58.456 --> 00:06:00.911
and then we kind of have
the same thing on the right,

00:06:00.911 --> 00:06:05.675
you know, boldfaced
vector x minus x naught,

00:06:05.675 --> 00:06:08.135
except that we transpose it,
we kind of put it on its side,

00:06:08.135 --> 00:06:08.968
and the way you denote that,

00:06:08.968 --> 00:06:11.465
you have a little T there, for transpose.

00:06:11.465 --> 00:06:15.698
So this term captures all
of the quadratic information

00:06:15.698 --> 00:06:18.566
that we need for the approximation.

00:06:18.566 --> 00:06:20.402
So just to put it all together,

00:06:20.402 --> 00:06:23.434
if we go back up and we put
the Constant term that we have,

00:06:23.434 --> 00:06:25.458
the Linear term, and this quadratic form

00:06:25.458 --> 00:06:27.569
that we just found, all together,

00:06:27.569 --> 00:06:30.953
what we get is that the
quadratic approximation of f,

00:06:30.953 --> 00:06:33.553
which is a function we'll
think of it as a vector input,

00:06:33.553 --> 00:06:37.915
boldfaced x, it equals the
function itself evaluated at,

00:06:37.915 --> 00:06:41.696
you know, whatever point
we're approximating near,

00:06:41.696 --> 00:06:44.215
plus the gradient of f, which is kind of,

00:06:44.215 --> 00:06:46.778
it's vector analog of a derivative,

00:06:46.778 --> 00:06:50.454
evaluated at that point, so
this is a constant vector,

00:06:50.454 --> 00:06:53.537
dot product with the variable vector,

00:06:55.357 --> 00:06:58.690
x minus the constant vector x naught,

00:07:00.082 --> 00:07:04.812
that whole thing, plus 1/2
the, then we'll just copy down

00:07:04.812 --> 00:07:06.878
this whole quadratic term up there,

00:07:06.878 --> 00:07:09.461
the variable minus the constant

00:07:13.253 --> 00:07:14.861
multiplied by the Hessian,

00:07:14.861 --> 00:07:16.263
which is kind of like an extension

00:07:16.263 --> 00:07:19.865
of the second derivative,
two multivariable functions,

00:07:19.865 --> 00:07:21.147
and we're evaluating that, no, let's see,

00:07:21.147 --> 00:07:24.041
we're evaluating it at the constant,

00:07:24.041 --> 00:07:28.217
at the constant, x naught,
and then on the right side,

00:07:28.217 --> 00:07:32.384
we're multiplying it by the
variable, x minus x naught.

00:07:35.496 --> 00:07:37.579
And this, this is the
quadratic approximation

00:07:37.579 --> 00:07:40.730
in vector form, and the
important part is, now,

00:07:40.730 --> 00:07:43.144
it doesn't just have to be
of a two variable input.

00:07:43.144 --> 00:07:44.896
You could imagine plugging
in a three variable input,

00:07:44.896 --> 00:07:47.529
or a four variable input, and
all of these terms make sense.

00:07:47.529 --> 00:07:49.856
You know, you take the gradient
of a four variable function,

00:07:49.856 --> 00:07:51.466
you'll get a vector with four components.

00:07:51.466 --> 00:07:53.816
You take the Hessian of
a four variable function,

00:07:53.816 --> 00:07:55.629
you would get a four by four matrix.

00:07:55.629 --> 00:07:57.401
And all of these terms make sense.

00:07:57.401 --> 00:07:59.185
And I think it's also
prettier to write it this way,

00:07:59.185 --> 00:08:01.465
because it looks a lot more
like a Taylor expansion

00:08:01.465 --> 00:08:03.493
in the single variable world.

00:08:03.493 --> 00:08:05.186
You have, you know, a constant term,

00:08:05.186 --> 00:08:08.746
plus the value of a derivative,
times x minus a constant,

00:08:08.746 --> 00:08:11.705
plus 1/2 what's kind of like
the second derivative term,

00:08:11.705 --> 00:08:13.445
what's kind of like taking an x squared,

00:08:13.445 --> 00:08:16.098
but this is how it looks
in the vector world.

00:08:16.098 --> 00:08:17.826
So in that way it's
actually maybe a little bit

00:08:17.826 --> 00:08:19.598
more familiar than writing it out

00:08:19.598 --> 00:08:22.630
in the full, you know,
component by component term,

00:08:22.630 --> 00:08:25.055
where it's easy to kind of
get lost in the weeds there.

00:08:25.055 --> 00:08:29.848
So, full vectorized form of
the quadratic approximation

00:08:29.848 --> 00:08:32.922
of a scalar valued multivariable function.

00:08:32.922 --> 00:08:34.295
Boy, is that a lot to say.


Expressing a quadratic form with a matrix

WEBVTT
Kind: captions
Language: en

00:00:00.702 --> 00:00:01.535
- [Voiceover] Hey guys.

00:00:01.535 --> 00:00:02.683
There's one more thing
I need to talk about

00:00:02.683 --> 00:00:05.179
before I can describe the vectorized form

00:00:05.179 --> 00:00:08.240
for the quadratic approximation
of multivariable functions

00:00:08.240 --> 00:00:09.848
which is a mouthful to say

00:00:09.848 --> 00:00:11.301
so let's say you have
some kind of expression

00:00:11.301 --> 00:00:14.046
that looks like a times x squared

00:00:14.046 --> 00:00:15.633
and I'm thinking x is a variable

00:00:15.633 --> 00:00:17.700
times b times xy,

00:00:17.700 --> 00:00:19.129
y is another variable,

00:00:19.129 --> 00:00:21.214
plus c times y squared

00:00:21.214 --> 00:00:23.298
and I'm thinking of a, b
and c as being constants

00:00:23.298 --> 00:00:25.433
and x and y as being variables.

00:00:25.433 --> 00:00:28.002
Now, this kind of
expression has a fancy name.

00:00:28.002 --> 00:00:30.687
It's called a quadratic form.

00:00:30.687 --> 00:00:31.520
Quadratic

00:00:32.789 --> 00:00:33.687
form.

00:00:33.687 --> 00:00:34.520
And that always threw me off.

00:00:34.520 --> 00:00:35.919
I always kind of was like, what,

00:00:35.919 --> 00:00:37.034
what does form mean?

00:00:37.034 --> 00:00:38.508
I know what a quadratic expression is

00:00:38.508 --> 00:00:40.552
and quadratic typically
means something is squared

00:00:40.552 --> 00:00:41.452
or you have two variables

00:00:41.452 --> 00:00:43.071
but why do they call it a form?

00:00:43.071 --> 00:00:45.584
And basically it just means
that the only things in here

00:00:45.584 --> 00:00:46.564
are quadratic.

00:00:46.564 --> 00:00:48.401
It's not the case that you have an x term

00:00:48.401 --> 00:00:50.770
sitting on its own or a constant out here

00:00:50.770 --> 00:00:52.896
like two when you're adding
all of those together

00:00:52.896 --> 00:00:55.676
instead it's just you have
purely quadratic terms

00:00:55.676 --> 00:00:57.390
but of course, mathematicians
don't want to call it

00:00:57.390 --> 00:00:59.023
just a purely quadratic expression

00:00:59.023 --> 00:01:00.738
instead they have to give
a fancy name to things

00:01:00.738 --> 00:01:03.475
so that it seems more
intimidating than it needs to be

00:01:03.475 --> 00:01:05.191
but anyways, so we have a quadratic form

00:01:05.191 --> 00:01:06.132
and the question is

00:01:06.132 --> 00:01:09.316
how can we express this
in a vectorized sense?

00:01:09.316 --> 00:01:11.240
And for analogy, let's
think about linear terms

00:01:11.240 --> 00:01:13.644
where let's say you have a times x

00:01:13.644 --> 00:01:15.451
plus b times y

00:01:15.451 --> 00:01:16.799
and I'll throw another variable in there,

00:01:16.799 --> 00:01:19.288
another constant times another variable z.

00:01:19.288 --> 00:01:20.434
If you see something like this

00:01:20.434 --> 00:01:23.456
where every variable is just
being multiplied by a constant

00:01:23.456 --> 00:01:25.622
and then you add terms
like that to each other,

00:01:25.622 --> 00:01:27.995
we can express this nicely with vectors

00:01:27.995 --> 00:01:30.940
where you pile all of the
constants into their own vector,

00:01:30.940 --> 00:01:33.316
a vector containing a, b and c

00:01:33.316 --> 00:01:34.992
and you imagine the dot product

00:01:34.992 --> 00:01:37.309
between that and a vector that contains

00:01:37.309 --> 00:01:38.755
all of the variable components,

00:01:38.755 --> 00:01:39.588
x, y and z

00:01:41.044 --> 00:01:43.536
and the convenience here is
then you can have just a symbol

00:01:43.536 --> 00:01:45.090
like a v let's say

00:01:45.090 --> 00:01:47.207
which represents this
whole constant vector

00:01:47.207 --> 00:01:48.107
and then you can write down,

00:01:48.107 --> 00:01:49.777
take the dot product between that

00:01:49.777 --> 00:01:51.531
and then have another symbol,

00:01:51.531 --> 00:01:53.204
maybe a bold faced x

00:01:53.204 --> 00:01:56.682
which represents a vector that
contains all of the variables

00:01:56.682 --> 00:01:58.683
and this way, your notation
just kind of looks like

00:01:58.683 --> 00:02:00.280
a constant times a variable

00:02:00.280 --> 00:02:01.672
just like in the single variable world

00:02:01.672 --> 00:02:03.915
when you have a constant
number times a variable number,

00:02:03.915 --> 00:02:05.915
it's kind of like taking a constant vector

00:02:05.915 --> 00:02:07.387
times a variable vector.

00:02:07.387 --> 00:02:09.148
And the importance of
writing things down like this

00:02:09.148 --> 00:02:10.338
is that v could be a vector

00:02:10.338 --> 00:02:11.817
that contains not just three numbers

00:02:11.817 --> 00:02:13.043
but a hundred numbers

00:02:13.043 --> 00:02:15.857
and then x would have a
hundred corresponding variables

00:02:15.857 --> 00:02:18.204
and the notation doesn't
become any more complicated.

00:02:18.204 --> 00:02:20.762
It's generalizable at
the higher dimensions.

00:02:20.762 --> 00:02:21.595
So the question is

00:02:21.595 --> 00:02:23.510
can be we do something similar like that

00:02:23.510 --> 00:02:24.952
with our quadratic form?

00:02:24.952 --> 00:02:25.785
Because you can imagine

00:02:25.785 --> 00:02:27.896
let's say we started
introducing the variable z

00:02:27.896 --> 00:02:29.662
then you would have to
have some other term,

00:02:29.662 --> 00:02:33.039
some other constant times
the xz quadratic term

00:02:33.039 --> 00:02:34.329
and then some other constant

00:02:34.329 --> 00:02:36.244
times the z squared quadratic term

00:02:36.244 --> 00:02:38.353
and another one for the yz quadratic term

00:02:38.353 --> 00:02:39.254
and it would get out of hand

00:02:39.254 --> 00:02:40.598
and as soon as you
start introducing things

00:02:40.598 --> 00:02:42.102
like a hundred variables,

00:02:42.102 --> 00:02:43.457
it would get seriously out of hand

00:02:43.457 --> 00:02:45.561
because there's a lot of
different quadratic terms

00:02:45.561 --> 00:02:47.748
so we want a nice way to express this.

00:02:47.748 --> 00:02:50.616
And I'm just going to kind
of show you how we do it

00:02:50.616 --> 00:02:53.575
and then we'll work it through
to see why it makes sense.

00:02:53.575 --> 00:02:56.052
So usually, instead of
thinking of b times xy,

00:02:56.052 --> 00:02:58.667
we actually think of this
as two times some constant

00:02:58.667 --> 00:02:59.733
times xy

00:02:59.733 --> 00:03:01.258
and this of course
doesn't make a difference.

00:03:01.258 --> 00:03:03.180
You would just change what b represents

00:03:03.180 --> 00:03:05.504
but you'll see why it's more
convenient to write it this way

00:03:05.504 --> 00:03:06.932
in just a moment.

00:03:06.932 --> 00:03:10.177
So the vectorized way to describe
a quadratic form like this

00:03:10.177 --> 00:03:12.204
is to take a matrix,

00:03:12.204 --> 00:03:14.531
a two by two matrix since
this is two dimensions

00:03:14.531 --> 00:03:16.899
where a and c are in the diagonal

00:03:16.899 --> 00:03:19.428
and then b is on the other diagonal

00:03:19.428 --> 00:03:21.675
and we always think of these
as being symmetric matrices

00:03:21.675 --> 00:03:23.623
so if you imagine kind of
reflecting the whole matrix

00:03:23.623 --> 00:03:25.034
about this line,

00:03:25.034 --> 00:03:25.928
you'll get the same number

00:03:25.928 --> 00:03:29.384
so it's important that we
have that kind of symmetry.

00:03:29.384 --> 00:03:32.053
And now what you do is
you multiply the vector,

00:03:32.053 --> 00:03:34.498
the variable vector that's got x, y

00:03:34.498 --> 00:03:36.627
on the right side of this matrix

00:03:36.627 --> 00:03:38.633
and then you multiply it again

00:03:38.633 --> 00:03:40.715
but you turn it on its side

00:03:40.715 --> 00:03:42.348
so instead of being a vertical vector,

00:03:42.348 --> 00:03:44.956
you transpose it to
being a horizontal vector

00:03:44.956 --> 00:03:46.263
on the other side.

00:03:46.263 --> 00:03:47.655
And this is a little bit analogous too

00:03:47.655 --> 00:03:49.655
having two variables multiplied in.

00:03:49.655 --> 00:03:52.472
You have two vectors multiplied
in but on either side.

00:03:52.472 --> 00:03:53.763
And this is a good point by the way

00:03:53.763 --> 00:03:56.391
if you are uncomfortable
with matrix multiplication

00:03:56.391 --> 00:03:57.739
to maybe pause the video,

00:03:57.739 --> 00:03:59.862
go find the videos about
matrix multiplication

00:03:59.862 --> 00:04:01.656
and kind of refresh or learn about that

00:04:01.656 --> 00:04:02.489
because moving forward,

00:04:02.489 --> 00:04:03.322
I'm just going to assume

00:04:03.322 --> 00:04:05.963
that it's something you're familiar with.

00:04:05.963 --> 00:04:08.435
So going about computing this,

00:04:08.435 --> 00:04:11.275
first, let's tackle this
right multiplication here.

00:04:11.275 --> 00:04:14.318
We have a matrix multiplied by a vector.

00:04:14.318 --> 00:04:16.503
Well, the first component that we get,

00:04:16.503 --> 00:04:18.465
we're going to multiply the top row

00:04:18.465 --> 00:04:19.992
by each corresponding term in the vector

00:04:19.992 --> 00:04:21.933
so it'll be a times x.

00:04:21.933 --> 00:04:22.766
a times x

00:04:24.052 --> 00:04:25.302
plus b times y.

00:04:26.218 --> 00:04:28.768
Plus b times that second term y

00:04:28.768 --> 00:04:30.359
and then similarly for the bottom term,

00:04:30.359 --> 00:04:31.997
we'll take the bottom row

00:04:31.997 --> 00:04:33.507
and multiply the corresponding terms

00:04:33.507 --> 00:04:35.144
so b times x.

00:04:35.144 --> 00:04:35.977
b times x

00:04:37.435 --> 00:04:38.685
plus c times y.

00:04:39.797 --> 00:04:40.630
c times y.

00:04:43.027 --> 00:04:43.860
So that's what it looks like

00:04:43.860 --> 00:04:45.480
when we do that right multiplication

00:04:45.480 --> 00:04:47.359
and of course we've got to
keep our transposed vector

00:04:47.359 --> 00:04:49.457
over there on the right,

00:04:49.457 --> 00:04:51.321
on the left side.

00:04:51.321 --> 00:04:52.629
So now, we have,

00:04:52.629 --> 00:04:54.225
this is just a two by one vector now

00:04:54.225 --> 00:04:56.385
and this is a one by two.

00:04:56.385 --> 00:04:57.978
You could think of it
as a horizontal vector

00:04:57.978 --> 00:05:00.214
or a one by two matrix

00:05:00.214 --> 00:05:02.172
but now when we multiply these guys,

00:05:02.172 --> 00:05:04.743
you just kind of line up
the corresponding terms.

00:05:04.743 --> 00:05:07.392
You'll have x multiplied by
that entire top expression

00:05:07.392 --> 00:05:10.903
so x multiplied by ax plus by.

00:05:10.903 --> 00:05:11.736
ax plus by

00:05:15.360 --> 00:05:17.602
and then we add that to the second term y

00:05:17.602 --> 00:05:19.239
multiplied by the second term of this guy

00:05:19.239 --> 00:05:20.790
which is bx plus cy

00:05:20.790 --> 00:05:22.290
so y multiplied by

00:05:23.812 --> 00:05:24.645
bx plus cy

00:05:27.734 --> 00:05:29.898
and all of these are numbers
so we can simplify it

00:05:29.898 --> 00:05:32.024
once we start distributing the first term

00:05:32.024 --> 00:05:33.782
is x times a times x

00:05:33.782 --> 00:05:35.449
so that's ax squared

00:05:36.884 --> 00:05:39.788
and then the next term
is x times b times y

00:05:39.788 --> 00:05:41.538
so that's b times xy.

00:05:43.666 --> 00:05:45.752
Over here, we have y times b times x

00:05:45.752 --> 00:05:47.526
so that's the same thing as b times xy

00:05:47.526 --> 00:05:49.061
so that's kind of why we have,

00:05:49.061 --> 00:05:50.778
why it's convenient to write a two there

00:05:50.778 --> 00:05:53.880
because that naturally
comes out of our expansion.

00:05:53.880 --> 00:05:56.777
And then the last term
is y times c times y

00:05:56.777 --> 00:05:58.527
so that's cy squared.

00:06:00.123 --> 00:06:02.299
So we get back the original quadratic form

00:06:02.299 --> 00:06:04.474
that we were shooting for.

00:06:04.474 --> 00:06:07.801
ax squared plus two bxy plus cy squared

00:06:07.801 --> 00:06:09.685
That's how this entire term expands.

00:06:09.685 --> 00:06:11.113
As you kind of work it through,

00:06:11.113 --> 00:06:14.384
you end up with the same
quadratic expression.

00:06:14.384 --> 00:06:17.403
Now, the convenience
of this quadratic form

00:06:17.403 --> 00:06:19.276
being written with a matrix like this

00:06:19.276 --> 00:06:20.847
is that we can write
this more abstractally

00:06:20.847 --> 00:06:23.439
and instead of writing
the whole matrix in,

00:06:23.439 --> 00:06:25.508
you could just let a letter like m

00:06:25.508 --> 00:06:26.964
represent that whole matrix

00:06:26.964 --> 00:06:29.267
and then take the vector
that represents the variable,

00:06:29.267 --> 00:06:30.982
maybe a bold faced x

00:06:30.982 --> 00:06:32.665
and you would multiply it on the right

00:06:32.665 --> 00:06:34.900
and then you transpose it
and multiply it on the left

00:06:34.900 --> 00:06:36.043
so typically you denote that

00:06:36.043 --> 00:06:38.447
by putting a little t as a superscript

00:06:38.447 --> 00:06:41.578
so x transposed multiplied by

00:06:41.578 --> 00:06:43.264
the matrix from the left

00:06:43.264 --> 00:06:45.007
and this expression,

00:06:45.007 --> 00:06:49.479
this is what a quadratic form
looks like in vectorized form

00:06:49.479 --> 00:06:50.499
and the convenience is the same

00:06:50.499 --> 00:06:51.973
as it was in the linear case.

00:06:51.973 --> 00:06:52.952
Just like v could represent

00:06:52.952 --> 00:06:54.954
something that had a hundred
different numbers in it

00:06:54.954 --> 00:06:56.994
and x would have a hundred
different constants,

00:06:56.994 --> 00:06:58.344
you could do something similar here

00:06:58.344 --> 00:07:01.285
where you can write that same expression

00:07:01.285 --> 00:07:03.449
even if the matrix m is super huge.

00:07:03.449 --> 00:07:04.512
Let's just see what this would look like

00:07:04.512 --> 00:07:07.045
in a three dimensional circumstance so,

00:07:07.045 --> 00:07:08.502
actually, I'll need more room

00:07:08.502 --> 00:07:10.072
so I'll go down even further.

00:07:10.072 --> 00:07:14.136
So we have x transpose
multiplied by the matrix

00:07:14.136 --> 00:07:16.597
multiplied by x, bold faced x

00:07:16.597 --> 00:07:18.573
and let's say instead this represented,

00:07:18.573 --> 00:07:20.906
you have x then y then z,

00:07:22.132 --> 00:07:23.719
our transposed vector

00:07:23.719 --> 00:07:24.986
and then our matrix,

00:07:24.986 --> 00:07:27.736
our matrix let's say was a, b, c,

00:07:28.748 --> 00:07:32.506
d, e, f and because it
needs to be symmetric,

00:07:32.506 --> 00:07:34.666
whatever term is in this spot here

00:07:34.666 --> 00:07:36.053
needs to be the same as over here

00:07:36.053 --> 00:07:38.180
kind of when you reflect
it about that diagonal.

00:07:38.180 --> 00:07:40.915
Similarly, c, that's going
to be the same term here

00:07:40.915 --> 00:07:42.525
and e would be over here.

00:07:42.525 --> 00:07:45.485
So there's only really six
free terms that you have

00:07:45.485 --> 00:07:47.529
but if fills up this entire matrix

00:07:47.529 --> 00:07:48.362
and then on the right side,

00:07:48.362 --> 00:07:50.025
we would multiply that by

00:07:50.025 --> 00:07:50.858
x, y, z.

00:07:54.220 --> 00:07:55.958
Now, I won't work it out in this video

00:07:55.958 --> 00:07:57.869
but you can imagine actually
multiplying this matrix

00:07:57.869 --> 00:07:58.901
by this vector

00:07:58.901 --> 00:08:01.388
and then multiplying the
corresponding vector that you get

00:08:01.388 --> 00:08:03.109
by this transposed vector

00:08:03.109 --> 00:08:04.994
and you'll get some kind of quadratic form

00:08:04.994 --> 00:08:06.143
with three variables

00:08:06.143 --> 00:08:08.427
and the point is you'll
get a very complicated one

00:08:08.427 --> 00:08:10.751
but it's very simple to
express things like this.

00:08:10.751 --> 00:08:11.811
So with that tool in hand,

00:08:11.811 --> 00:08:12.644
in the next video,

00:08:12.644 --> 00:08:14.629
I will talk about how
we can use this notation

00:08:14.629 --> 00:08:16.957
to express the quadratic approximations

00:08:16.957 --> 00:08:18.438
for multivariable functions.

00:08:18.438 --> 00:08:19.271
See you then.


Local linearization

WEBVTT
Kind: captions
Language: en

00:00:00.000 --> 00:00:01.402
- [Voiceover] In the last couple videos, I

00:00:01.402 --> 00:00:03.896
showed how you can take a function, ah,

00:00:03.896 --> 00:00:06.379
just a function with two inputs,

00:00:06.379 --> 00:00:09.015
and find the tangent plane to its graph,

00:00:09.015 --> 00:00:10.328
and the way that you think about this,

00:00:10.328 --> 00:00:11.581
you first find a point,

00:00:11.581 --> 00:00:12.994
some kind of input point,

00:00:12.994 --> 00:00:14.757
which is, you know I'll just write

00:00:14.757 --> 00:00:17.046
abstractly as x nought and y nought.

00:00:17.046 --> 00:00:19.471
And you see where that
point ends up on the graph,

00:00:19.471 --> 00:00:21.282
and you wanna find a new function,

00:00:21.282 --> 00:00:23.462
a new function which we were calling L,

00:00:23.462 --> 00:00:24.894
and maybe you say L sub f,

00:00:24.894 --> 00:00:27.158
which also is a function of x and y.

00:00:27.158 --> 00:00:28.657
And you want the graph of that function

00:00:28.657 --> 00:00:31.135
to be a plane tangent to the graph.

00:00:31.135 --> 00:00:33.428
Now this often goes by another name.

00:00:33.428 --> 00:00:36.481
This will go under the
name Local Linearization,

00:00:36.481 --> 00:00:40.648
Local linearization, this is
kind of a long word, zation.

00:00:43.013 --> 00:00:44.933
And what this basically means,

00:00:44.933 --> 00:00:46.674
the word local means you're looking at

00:00:46.674 --> 00:00:47.890
a specific input point.

00:00:47.890 --> 00:00:48.893
So in this case,

00:00:48.893 --> 00:00:51.955
it's a specific input
point x nought, y nought,

00:00:51.955 --> 00:00:54.232
and the idea of a linearization,

00:00:54.232 --> 00:00:56.621
a linearization, means
you're approximating

00:00:56.621 --> 00:00:58.676
the function with something simpler,

00:00:58.676 --> 00:01:00.243
with something that's actually linear,

00:01:00.243 --> 00:01:01.475
and I'll tell you what
I mean by linear in,

00:01:01.475 --> 00:01:02.738
in just a moment.

00:01:02.738 --> 00:01:05.997
But the whole idea here is that
we don't really care about,

00:01:05.997 --> 00:01:08.939
you know, tangent planes
in an abstract 3D spaced

00:01:08.939 --> 00:01:10.238
to some kind of graph.

00:01:10.238 --> 00:01:11.620
The whole reason for doing this,

00:01:11.620 --> 00:01:14.266
is that this is a really good
way to approximate a function,

00:01:14.266 --> 00:01:16.393
which is potentially a
very complicated function

00:01:16.393 --> 00:01:17.744
with something that's much easier,

00:01:17.744 --> 00:01:20.114
something that has constant
partial derivatives.

00:01:20.114 --> 00:01:22.147
Now my goal of this
video is gonna be to show

00:01:22.147 --> 00:01:24.569
how we write this local linearization here

00:01:24.569 --> 00:01:27.370
in vector form, because
it'll be both more compact,

00:01:27.370 --> 00:01:28.907
and hopefully easier to remember,

00:01:28.907 --> 00:01:30.136
and also it's more general.

00:01:30.136 --> 00:01:32.071
It'll apply to things
that have more than just,

00:01:32.071 --> 00:01:34.691
just two input variables
like this one does.

00:01:34.691 --> 00:01:36.422
So just to remind us of where we were,

00:01:36.422 --> 00:01:38.816
and what we got to in
the last couple videos,

00:01:38.816 --> 00:01:40.579
I'll write a little bit
more abstractly this time,

00:01:40.579 --> 00:01:42.345
rather than a specific example.

00:01:42.345 --> 00:01:44.640
The way you do this local linearization

00:01:44.640 --> 00:01:46.297
is first you find the partial derivative

00:01:46.297 --> 00:01:47.801
of f with respect to x,

00:01:47.801 --> 00:01:51.444
which I'll write with
the subscript notation.

00:01:51.444 --> 00:01:54.679
And you evaluate that at x of o

00:01:54.679 --> 00:01:55.964
or x nought, y nought.

00:01:55.964 --> 00:01:57.177
You evaluate it at the point

00:01:57.177 --> 00:01:59.004
about which you're approximating

00:01:59.004 --> 00:02:03.447
and then you multiply that
by x minus that constant.

00:02:03.447 --> 00:02:04.806
So the only variable right here,

00:02:04.806 --> 00:02:05.757
everything is a constant,

00:02:05.757 --> 00:02:08.028
but the only variable part is that x.

00:02:08.028 --> 00:02:09.007
And then we add to that,

00:02:09.007 --> 00:02:10.502
basically doing the same thing with y.

00:02:10.502 --> 00:02:12.713
You take the partial
derivative with respect to y,

00:02:12.713 --> 00:02:14.749
you evaluate it at the input point,

00:02:14.749 --> 00:02:17.083
the point about which you are linearizing,

00:02:17.083 --> 00:02:20.750
and then you multiply
it by y minus ys of o.

00:02:22.214 --> 00:02:23.938
And then to this entire thing

00:02:23.938 --> 00:02:25.720
because you wanna make sure that when

00:02:25.720 --> 00:02:28.153
you evaluate this function
at the input point itself.

00:02:28.153 --> 00:02:30.457
You see, when you plug
in x nought and y nought,

00:02:30.457 --> 00:02:31.585
this terms goes to zero,

00:02:31.585 --> 00:02:33.449
cause x nought minus x nought is zero.

00:02:33.449 --> 00:02:34.532
This terms goes to zero,

00:02:34.532 --> 00:02:36.132
and this is the whole
reason we kind of paired

00:02:36.132 --> 00:02:38.864
up these terms and organized
the constants in this way.

00:02:38.864 --> 00:02:40.518
This way, you can just think about adding

00:02:40.518 --> 00:02:44.361
whatever the function itself
evaluates to at that point.

00:02:44.361 --> 00:02:46.090
And this will ensure
that your linearization

00:02:46.090 --> 00:02:48.212
actually equals the function itself

00:02:48.212 --> 00:02:49.097
at the local point.

00:02:49.097 --> 00:02:51.379
Cause hopefully if you're
approximating it near a point,

00:02:51.379 --> 00:02:53.323
then at that point, it's actually equal.

00:02:53.323 --> 00:02:56.250
So what do I mean by this word linear?

00:02:56.250 --> 00:02:58.550
The word linear has a
very precise formulation,

00:02:58.550 --> 00:03:00.731
especially in the context
of linear algebra,

00:03:00.731 --> 00:03:02.363
and admittedly, this is not actually

00:03:02.363 --> 00:03:04.709
a linear function in the technical sense.

00:03:04.709 --> 00:03:05.828
But loosely what it means,

00:03:05.828 --> 00:03:07.527
and the reason people call it linear,

00:03:07.527 --> 00:03:10.059
is that this x term
here, this variable term,

00:03:10.059 --> 00:03:11.869
doesn't have anything
fancy going on with it.

00:03:11.869 --> 00:03:13.984
It's just being multiplied by a constant,

00:03:13.984 --> 00:03:15.829
and similarly this y term it's just being

00:03:15.829 --> 00:03:16.936
multiplied by a constant.

00:03:16.936 --> 00:03:18.588
It's not squared, there's no square root,

00:03:18.588 --> 00:03:20.939
it's not in an exponent
or anything like that.

00:03:20.939 --> 00:03:23.000
And although there is a
more technical meaning

00:03:23.000 --> 00:03:24.045
of the word linear,

00:03:24.045 --> 00:03:25.497
this is all it really
needs in this context.

00:03:25.497 --> 00:03:26.719
This is all you need to think about.

00:03:26.719 --> 00:03:30.065
Each variable is just
multiplied by a constant.

00:03:30.065 --> 00:03:32.306
Now you might see this in
a more complicated form,

00:03:32.306 --> 00:03:36.148
or what's at first a more
complicated form using vectors.

00:03:36.148 --> 00:03:37.593
So first of all, let's think about

00:03:37.593 --> 00:03:38.975
how we would start describing

00:03:38.975 --> 00:03:41.330
everything going on here with vectors.

00:03:41.330 --> 00:03:42.638
So the input, ah,

00:03:42.638 --> 00:03:45.070
rather than talk about the
input as being a pair of points,

00:03:45.070 --> 00:03:47.583
what I wanna say is that
there's some vector,

00:03:47.583 --> 00:03:49.165
some vector that has
these as its components,

00:03:49.165 --> 00:03:51.207
and we just wanna capture that all,

00:03:51.207 --> 00:03:52.696
and I wanna give that a name.

00:03:52.696 --> 00:03:54.949
And kind of unfortunately
the name that we give this,

00:03:54.949 --> 00:03:57.173
it's very common to just call it x,

00:03:57.173 --> 00:03:59.009
and maybe a bold-faced x and that would be

00:03:59.009 --> 00:04:00.689
easier to do typing than it is writing,

00:04:00.689 --> 00:04:02.234
so I'll just kind of try
to emphasize bold-faced

00:04:02.234 --> 00:04:03.681
x equals this vector,

00:04:03.681 --> 00:04:05.213
and where's, that's confusing cause x

00:04:05.213 --> 00:04:08.056
is already one of the input
variables that's just a number.

00:04:08.056 --> 00:04:09.303
Um, but I'll try to emphasize it,

00:04:09.303 --> 00:04:10.165
just making it bold.

00:04:10.165 --> 00:04:11.555
You'll see this in writing a lot.

00:04:11.555 --> 00:04:13.069
X is this input vector,

00:04:13.069 --> 00:04:15.861
and then similarly, the specified, ah,

00:04:15.861 --> 00:04:18.344
specified input about
which we are approximating,

00:04:18.344 --> 00:04:20.330
you would call, see I'll make it a nice

00:04:20.330 --> 00:04:22.196
bold-faced x nought.

00:04:22.196 --> 00:04:24.829
We'll, we'll do that nought
to just kind of indicate

00:04:24.829 --> 00:04:26.584
that it's a constant of some kind.

00:04:26.584 --> 00:04:27.417
And what that is,

00:04:27.417 --> 00:04:29.184
it's a vector containing the two numbers

00:04:29.184 --> 00:04:30.434
x nought, y nought.

00:04:31.649 --> 00:04:32.482
So this is just us
starting to write things

00:04:32.482 --> 00:04:34.438
in a more vectorized way

00:04:34.438 --> 00:04:36.093
and the convenience here is that,

00:04:36.093 --> 00:04:37.248
that if you're dealing with a function

00:04:37.248 --> 00:04:40.436
with three input variables
or four or a hundred,

00:04:40.436 --> 00:04:41.790
you could still just write it as this

00:04:41.790 --> 00:04:44.030
bold-faced x with the understanding

00:04:44.030 --> 00:04:46.125
that the vector has a lot more components.

00:04:46.125 --> 00:04:48.185
So now, let's take a look at,

00:04:48.185 --> 00:04:51.645
at these first two terms
in our linearization.

00:04:51.645 --> 00:04:53.695
We can start thinking of this as a,

00:04:53.695 --> 00:04:55.874
as a dot product, actually.

00:04:55.874 --> 00:04:57.462
So let me first just kind of move this guy

00:04:57.462 --> 00:04:59.226
out of the way and give
ourselves some room.

00:04:59.226 --> 00:05:01.457
So he's gonna just go up there,

00:05:01.457 --> 00:05:03.114
this the same guy,

00:05:03.114 --> 00:05:05.642
and now I wanna think about
writing this other term here

00:05:05.642 --> 00:05:07.319
as a, as a dot product.

00:05:07.319 --> 00:05:10.004
And what that looks like is we have

00:05:10.004 --> 00:05:10.837
the two partial derivatives fs of x

00:05:10.837 --> 00:05:14.686
and fs of y, indicating
the partial derivatives

00:05:14.686 --> 00:05:16.660
with respect to x and y,

00:05:16.660 --> 00:05:18.517
and each one of them is evaluated.

00:05:18.517 --> 00:05:19.666
Let's see, I'll do it, I'll do it

00:05:19.666 --> 00:05:22.916
evaluating at our bold-faced x nought,

00:05:23.898 --> 00:05:25.370
and then this one is also evaluated

00:05:25.370 --> 00:05:27.128
at that bold-faced x nought.

00:05:27.128 --> 00:05:29.141
So, really you're thinking
about this as being,

00:05:29.141 --> 00:05:31.431
you know, a vector that contains
two different variables.

00:05:31.431 --> 00:05:34.292
You're just packing it
into a single symbol,

00:05:34.292 --> 00:05:35.860
and the dot product here

00:05:35.860 --> 00:05:38.669
is against, ah, you
know, the first component

00:05:38.669 --> 00:05:40.408
is x minus x nought,

00:05:40.408 --> 00:05:44.295
so I'd write that as x
minus x nought the number,

00:05:44.295 --> 00:05:45.720
and then similarly, y minus,

00:05:45.720 --> 00:05:47.658
let's see, I'll do it in the same color,

00:05:47.658 --> 00:05:49.158
y nought the number.

00:05:50.584 --> 00:05:52.287
Ah, but we can write each one of these

00:05:52.287 --> 00:05:53.965
in a more compact form,

00:05:53.965 --> 00:05:56.605
where this, the vector that
has the partial derivatives,

00:05:56.605 --> 00:05:59.164
that's the gradient, and
if that feels unfamiliar,

00:05:59.164 --> 00:06:01.453
maybe go back and check out
the videos on the gradient,

00:06:01.453 --> 00:06:03.014
but this whole vector is
basically just saying,

00:06:03.014 --> 00:06:05.089
take the gradient and evaluate it at that,

00:06:05.089 --> 00:06:08.924
that vector input, you know, x nought.

00:06:08.924 --> 00:06:11.651
And in the second component here,

00:06:11.651 --> 00:06:13.035
that's telling you you've got

00:06:13.035 --> 00:06:16.026
x and y minus x nought and y nought.

00:06:16.026 --> 00:06:17.729
So what you're basically doing is taking

00:06:17.729 --> 00:06:19.501
the, you know, bold-faced input,

00:06:19.501 --> 00:06:21.933
the variable vector x,

00:06:21.933 --> 00:06:23.571
and then you're subtracting off,

00:06:23.571 --> 00:06:28.499
you know, x nought, where x
nought is some kind of constant.

00:06:28.499 --> 00:06:29.379
So this right here,

00:06:29.379 --> 00:06:31.416
this is just vector terms
where you're thinking of this

00:06:31.416 --> 00:06:32.872
as being a vector with two components,

00:06:32.872 --> 00:06:34.872
and this one is a vector
with two components,

00:06:34.872 --> 00:06:35.705
but if your function happened to

00:06:35.705 --> 00:06:37.009
be something more complicated,

00:06:37.009 --> 00:06:39.343
with, you know, a hundred input variables,

00:06:39.343 --> 00:06:40.914
this would be the same
thing you write down.

00:06:40.914 --> 00:06:43.067
You would just understand
that when you expand this,

00:06:43.067 --> 00:06:43.900
there's gonna be a hundred different

00:06:43.900 --> 00:06:45.740
components in the vector.

00:06:45.740 --> 00:06:47.860
Um, and this is what a linear term

00:06:47.860 --> 00:06:49.605
looks like in vector terminology,

00:06:49.605 --> 00:06:51.322
cause this dot product is telling you

00:06:51.322 --> 00:06:52.878
that all of the components

00:06:52.878 --> 00:06:54.218
of that bold-faced x vector,

00:06:54.218 --> 00:06:56.536
the, that expands into, you know,

00:06:56.536 --> 00:06:59.777
not bold-faced x, y, z,
whatever else it expands to.

00:06:59.777 --> 00:07:01.372
All of those are just being multiplied

00:07:01.372 --> 00:07:03.547
by some kind of constant.

00:07:03.547 --> 00:07:04.681
So we take that whole thing,

00:07:04.681 --> 00:07:08.317
that's, that's how you simplify
the first couple terms here,

00:07:08.317 --> 00:07:10.008
and of course, we just add on

00:07:10.008 --> 00:07:11.739
the value of the function itself.

00:07:11.739 --> 00:07:14.103
So you would take that as the linear term.

00:07:14.103 --> 00:07:15.937
And no, I kind of like to
add it on to the front,

00:07:15.937 --> 00:07:17.406
actually, where you think about taking

00:07:17.406 --> 00:07:19.420
the function itself and evaluating it

00:07:19.420 --> 00:07:23.210
at that, that constant input x nought,

00:07:23.210 --> 00:07:25.008
cause that way you can kind of think

00:07:25.008 --> 00:07:28.184
this is your constant term,
this is your constant term,

00:07:28.184 --> 00:07:31.274
and then the rest of the stuff
here is your linear term.

00:07:31.274 --> 00:07:32.299
Rest of your stuff is your linear.

00:07:32.299 --> 00:07:34.357
Cause later on if we
start adding other terms

00:07:34.357 --> 00:07:35.280
like a quadratic term

00:07:35.280 --> 00:07:36.344
or more complicated things,

00:07:36.344 --> 00:07:38.299
you can kind of keep
adding them on the end.

00:07:38.299 --> 00:07:39.536
So this right here,

00:07:39.536 --> 00:07:41.083
is the expression that you will often see

00:07:41.083 --> 00:07:43.279
for the local linearization.

00:07:43.279 --> 00:07:45.839
And the only place where the
actual variable shows up,

00:07:45.839 --> 00:07:48.801
the variable vector, is
right here, is this guy.

00:07:48.801 --> 00:07:50.539
Cause, you know, when
you evaluate the function

00:07:50.539 --> 00:07:52.976
f at a specified input,
that's just a constant.

00:07:52.976 --> 00:07:54.662
when you evaluate the
gradient at that input,

00:07:54.662 --> 00:07:55.768
it's just a constant.

00:07:55.768 --> 00:07:57.777
And we're subtracting off that, that

00:07:57.777 --> 00:07:59.868
um, specified input
that's just a constant.

00:07:59.868 --> 00:08:02.269
So this is the only place
where your variable shows up.

00:08:02.269 --> 00:08:03.657
So once all is said and done,

00:08:03.657 --> 00:08:05.178
and once you do your computations,

00:08:05.178 --> 00:08:07.208
this is a very simple function.

00:08:07.208 --> 00:08:09.406
And the, the important part
is maybe this is much simpler

00:08:09.406 --> 00:08:11.054
than the function f itself,

00:08:11.054 --> 00:08:12.749
which allows you to, you know,

00:08:12.749 --> 00:08:14.539
maybe compute something more quickly

00:08:14.539 --> 00:08:17.064
if you're writing a program that needs to,

00:08:17.064 --> 00:08:19.314
you know, deal with some
kind of complicated function,

00:08:19.314 --> 00:08:21.301
but runtime is an issue or maybe,

00:08:21.301 --> 00:08:23.101
it's a function that you
never knew in the first place,

00:08:23.101 --> 00:08:25.700
but you were able to approximate
its value at a point,

00:08:25.700 --> 00:08:27.207
and approximate its gradient.

00:08:27.207 --> 00:08:28.647
So this is what lets you approximate

00:08:28.647 --> 00:08:30.758
the function as a whole near that point.

00:08:30.758 --> 00:08:32.318
So again, this might look very abstract,

00:08:32.318 --> 00:08:34.235
but if you just kind of unravel everything

00:08:34.235 --> 00:08:35.693
and think back to where it came from

00:08:35.693 --> 00:08:37.657
and look at, look at the specific example

00:08:37.657 --> 00:08:39.382
of a, you know, tangent plane, um,

00:08:39.382 --> 00:08:41.478
hopefully it all makes
a little bit of sense

00:08:41.478 --> 00:08:44.007
and you see that this is
really just the simplest

00:08:44.007 --> 00:08:47.417
possible function that
evaluates to the same value

00:08:47.417 --> 00:08:49.436
as f when you input this point,

00:08:49.436 --> 00:08:51.554
and whose partial derivatives all evaluate

00:08:51.554 --> 00:08:53.197
to the same values as those of f

00:08:53.197 --> 00:08:55.028
at that specified point.

00:08:55.028 --> 00:08:56.510
And if you wanna see
more examples of this,

00:08:56.510 --> 00:08:57.998
and what it looks like and
maybe how you can use it

00:08:57.998 --> 00:09:00.047
to approximate certain functions,

00:09:00.047 --> 00:09:01.673
I have an article on that,
that you can go check out,

00:09:01.673 --> 00:09:03.780
and it would be
particularly good to kind of

00:09:03.780 --> 00:09:04.846
go in with a piece of paper

00:09:04.846 --> 00:09:07.476
and sort of work through
the examples yourself

00:09:07.476 --> 00:09:09.489
as you, as you work through it.

00:09:09.489 --> 00:09:10.609
And with that said,

00:09:10.609 --> 00:09:12.097
I will see you next video.


What is a tangent plane

WEBVTT
Kind: captions
Language: en

00:00:00.000 --> 00:00:01.182
- [Voiceover] Hi everyone.

00:00:01.182 --> 00:00:03.326
Here and in the next few videos
I'm gonna be talking about

00:00:03.326 --> 00:00:07.493
tangent planes of graphs,
and I'll specify this is

00:00:10.287 --> 00:00:12.338
tangent planes of graphs
and not of some other thing

00:00:12.338 --> 00:00:14.780
because in different context
of multivariable calculus you

00:00:14.780 --> 00:00:17.728
might be taking a tangent plane
of say a parametric surface

00:00:17.728 --> 00:00:20.975
or something like that but here
I'm just focused on graphs.

00:00:20.975 --> 00:00:24.586
In the single variable
world a common problem that

00:00:24.586 --> 00:00:26.673
people like to ask in calculus
is you have some sort of

00:00:26.673 --> 00:00:30.736
curve and you wanna find at a
given point what the tangent

00:00:30.736 --> 00:00:33.520
line to that curve is,
where the tangent line is.

00:00:33.520 --> 00:00:35.560
You’ll find the equation
for that tangent line

00:00:35.560 --> 00:00:37.708
and this gives you various
information how to,

00:00:37.708 --> 00:00:40.305
let's say you wanted to approximate
the function around that

00:00:40.305 --> 00:00:42.940
point and it turns out to be
a nice simple approximation.

00:00:42.940 --> 00:00:45.649
In the multivariable world
it's actually pretty similar

00:00:45.649 --> 00:00:48.547
in terms of geometric intuition
it's almost identical.

00:00:48.547 --> 00:00:50.771
You have some kind of graph of a function,

00:00:50.771 --> 00:00:52.904
like the one that I have
here, and then instead of

00:00:52.904 --> 00:00:55.420
having a tangent line,
because the line is a very

00:00:55.420 --> 00:00:57.687
one-dimensional thing and here
it's a very two-dimensional

00:00:57.687 --> 00:01:02.026
surface, instead you’ll have
some kind of tangent plane.

00:01:02.026 --> 00:01:04.554
This is something where it's
just gonna be barely kissing

00:01:04.554 --> 00:01:06.426
the graph in the same way
that the tangent line just

00:01:06.426 --> 00:01:10.042
barely kisses the function
graph in the one-dimensional

00:01:10.042 --> 00:01:12.592
circumstance, and it could be
at various different points

00:01:12.592 --> 00:01:14.256
rather than just being at that point.

00:01:14.256 --> 00:01:17.329
You could move it around and
say that it will just barely be

00:01:17.329 --> 00:01:21.166
kissing the graph of this
function but at different points.

00:01:21.166 --> 00:01:23.444
Usually the way that a problem
like this will be framed

00:01:23.444 --> 00:01:27.511
if you're trying to find such
a tangent plane is first,

00:01:27.511 --> 00:01:30.820
you think about the specified
input that you want.

00:01:30.820 --> 00:01:32.773
In the same way that over
in the single variable world

00:01:32.773 --> 00:01:36.397
what you might do is say,
"What is the input value here?"

00:01:36.397 --> 00:01:39.183
Maybe you'd name it like x
sub 0, and then you're gonna

00:01:39.183 --> 00:01:42.190
find the graph of the
function that corresponds to

00:01:42.190 --> 00:01:44.732
just kissing the graph
at that input point.

00:01:44.732 --> 00:01:49.102
Over here in the multivariable
world, move things about,

00:01:49.102 --> 00:01:51.942
you'll choose some input
point like this little red dot

00:01:51.942 --> 00:01:54.035
and that could be at
various different spots,

00:01:54.035 --> 00:01:55.091
it doesn’t have to be where I put it,

00:01:55.091 --> 00:01:56.827
you could imagine putting
it somewhere else.

00:01:56.827 --> 00:02:00.232
Once you decide on what
input point you want,

00:02:00.232 --> 00:02:02.252
you see where that is on the graph,

00:02:02.252 --> 00:02:04.679
so we go and say, "That
input point corresponds to

00:02:04.679 --> 00:02:06.570
such and such a height,"
so in this case it actually

00:02:06.570 --> 00:02:09.409
looks like the graph is
about zero at that point

00:02:09.409 --> 00:02:11.765
so the output of the
function would be zero.

00:02:11.765 --> 00:02:13.188
What you want is the plane

00:02:13.188 --> 00:02:15.414
that's tangent right at that point.

00:02:15.414 --> 00:02:16.552
You’ll draw some kind of plane that's

00:02:16.552 --> 00:02:18.885
tangent right at that point.

00:02:19.817 --> 00:02:22.504
If we think about what this
inner point corresponds to

00:02:22.504 --> 00:02:24.655
it's not x sub 0, a single variable input

00:02:24.655 --> 00:02:26.842
like we have in the single variable world,

00:02:26.842 --> 00:02:29.339
but instead that red dot
that you're seeing is gonna

00:02:29.339 --> 00:02:33.506
correspond to some kind of
input here, x sub 0, y sub 0.

00:02:35.414 --> 00:02:37.537
The ultimate goal over
here in our multivariable

00:02:37.537 --> 00:02:40.199
circumstance is gonna be to
find some kind of new function,

00:02:40.199 --> 00:02:43.198
so I'll write it down here,
some kind of new function

00:02:43.198 --> 00:02:47.299
that I'll call L, for linear,
that's gonna take in x and y,

00:02:47.299 --> 00:02:50.001
and we want the graph of that
function to be this plane,

00:02:50.001 --> 00:02:53.112
and you might specify that this
is depended on the original

00:02:53.112 --> 00:02:55.500
function that you have and
maybe also specify that it's

00:02:55.500 --> 00:02:58.454
depended on this input point
in some way, but the basic idea

00:02:58.454 --> 00:03:00.901
is we're gonna be looking
for a function whose graph

00:03:00.901 --> 00:03:03.554
is this plane tangent at a given point.

00:03:03.554 --> 00:03:05.552
In the next couple of videos
I'm gonna talk through

00:03:05.552 --> 00:03:06.649
how you actually compute that.

00:03:06.649 --> 00:03:08.789
It might seem a little intimidating
at first because how do

00:03:08.789 --> 00:03:11.880
you control a plane in
three dimensions like this?

00:03:11.880 --> 00:03:13.790
It's actually very similar
to the single variable

00:03:13.790 --> 00:03:17.548
circumstance, and you just
take it one step at a time.

00:03:17.548 --> 00:03:19.381
See you in next video.