Gradient and graphs

WEBVTT
Kind: captions
Language: en

00:00:00.863 --> 00:00:01.803
- [Voiceover] So here I'd like to talk

00:00:01.803 --> 00:00:03.487
about what the gradient means

00:00:03.487 --> 00:00:06.041
in the context of the graph of a function.

00:00:06.041 --> 00:00:09.134
So in the last video,
I defined the gradient,

00:00:09.134 --> 00:00:10.680
but let me just take a function here.

00:00:10.680 --> 00:00:15.521
And the one that I had graphed
is x-squared plus y-squared,

00:00:15.521 --> 00:00:17.681
f of x, y, equals
x-squared plus y-squared.

00:00:17.681 --> 00:00:19.886
So two-dimensional input,
which we think about

00:00:19.886 --> 00:00:22.325
as being kind of the xy-plane,

00:00:22.325 --> 00:00:24.530
and then a one-dimensional
output that's just the height

00:00:24.530 --> 00:00:26.748
of the graph above that plane.

00:00:26.748 --> 00:00:29.430
And I defined in the
last video, the gradient,

00:00:29.430 --> 00:00:31.287
to be a certain operator.

00:00:31.287 --> 00:00:33.168
An operator just means
you've taken a function

00:00:33.168 --> 00:00:34.527
and you output another function,

00:00:34.527 --> 00:00:37.545
and we use this upside down triangle.

00:00:37.545 --> 00:00:40.483
So it gives you another
function that's also of x and y,

00:00:40.483 --> 00:00:43.548
but this time it has a
vector valued output.

00:00:43.548 --> 00:00:45.823
And the two components of its output

00:00:45.823 --> 00:00:50.823
are the partial derivatives,
partial of f with respect to x,

00:00:51.326 --> 00:00:56.326
and the partial of f with respect to y.

00:00:58.154 --> 00:01:01.173
So for a function like this,
we actually evaluated it.

00:01:01.173 --> 00:01:03.020
Let's take a look.

00:01:03.020 --> 00:01:06.010
The first one is taking the derivative

00:01:06.010 --> 00:01:08.553
with respect to x, so
it looks at x and says,

00:01:08.553 --> 00:01:10.207
"You look like a variable to me.

00:01:10.207 --> 00:01:13.748
"I'm gonna take your derivative, your 2x."

00:01:13.748 --> 00:01:18.029
2x, but the y component
just looks like a constant

00:01:18.029 --> 00:01:20.073
as far as the partial x is concerned.

00:01:20.073 --> 00:01:22.789
And the derivative of a constant is zero.

00:01:22.789 --> 00:01:24.113
But when you take the partial derivative

00:01:24.113 --> 00:01:26.005
with respect to y, things reverse.

00:01:26.005 --> 00:01:27.364
It looks at the x component and says,

00:01:27.364 --> 00:01:28.443
"You look like a constant.

00:01:28.443 --> 00:01:29.872
"Your derivative is zero."

00:01:29.872 --> 00:01:31.353
But it looks at the y component and says,

00:01:31.353 --> 00:01:32.874
"Ah, you look like a variable.

00:01:32.874 --> 00:01:36.636
"Your derivative is 2y."

00:01:36.636 --> 00:01:38.586
So this ultimate function
we get, the gradient,

00:01:38.586 --> 00:01:40.595
which takes in a two variable input, xy,

00:01:40.595 --> 00:01:41.767
some point on this plane,

00:01:41.767 --> 00:01:43.149
but outputs a vector,

00:01:43.149 --> 00:01:45.448
can nicely be visualized
with a vector field.

00:01:45.448 --> 00:01:46.876
And I have another video on vector fields

00:01:46.876 --> 00:01:48.130
if you're feeling unsure.

00:01:48.130 --> 00:01:50.893
But I want you to just take a
moment, pause if you need to,

00:01:50.893 --> 00:01:54.387
and guess, or try to think
about what vector field

00:01:54.387 --> 00:01:55.351
this will look like.

00:01:55.351 --> 00:01:56.210
I'm gonna show you in a moment,

00:01:56.210 --> 00:01:57.232
but what's it gonna look like,

00:01:57.232 --> 00:02:01.295
the one that takes in
xy and outputs 2x, 2y?

00:02:01.295 --> 00:02:02.189
Alright, have you done it,

00:02:02.189 --> 00:02:05.835
have you thought about
what it's gonna look like?

00:02:05.835 --> 00:02:07.414
Here's what we get.

00:02:07.414 --> 00:02:10.630
It's a bunch of vectors
pointing away from the origin.

00:02:10.630 --> 00:02:12.290
And the basic reason for that

00:02:12.290 --> 00:02:14.716
is that if you have any given input point,

00:02:14.716 --> 00:02:18.208
and say it's got coordinates x, y,

00:02:18.208 --> 00:02:20.855
then the vector that that
input point represents

00:02:20.855 --> 00:02:22.812
would, you know, if it
went from the origin here,

00:02:22.812 --> 00:02:24.194
that's what that vector looks like,

00:02:24.194 --> 00:02:26.655
but the output is two times that vector.

00:02:26.655 --> 00:02:29.627
So when we attach that
output to the original point,

00:02:29.627 --> 00:02:31.787
we get something that's two
times that original vector

00:02:31.787 --> 00:02:33.296
but pointing in the same direction,

00:02:33.296 --> 00:02:34.770
which is away from the origin.

00:02:34.770 --> 00:02:37.789
We kind of drew it poorly here.

00:02:37.789 --> 00:02:39.805
And of course, when we draw vector fields,

00:02:39.805 --> 00:02:41.603
we don't usually draw them to scale.

00:02:41.603 --> 00:02:43.101
You scale them down

00:02:43.101 --> 00:02:44.680
just so that things
don't look as cluttered.

00:02:44.680 --> 00:02:46.479
That's why everything here,
they all look the same length,

00:02:46.479 --> 00:02:48.023
but color indicates length.

00:02:48.023 --> 00:02:50.183
So you should think of these
red guys as being really long,

00:02:50.183 --> 00:02:52.365
the blue ones as being really short.

00:02:52.365 --> 00:02:55.303
So what does this have to do
with the graph of the function?

00:02:55.303 --> 00:02:57.718
There's actually a really
cool interpretation.

00:02:57.718 --> 00:03:00.725
So imagine that you are just
walking along this graph,

00:03:00.725 --> 00:03:03.198
you know, you're a hiker
and this is a mountain.

00:03:03.198 --> 00:03:06.239
And you picture yourself at
any old point on this graph,

00:03:06.239 --> 00:03:09.281
let's say, what color should I use?

00:03:09.281 --> 00:03:11.859
Let's say you're sitting
at a point like this.

00:03:11.859 --> 00:03:13.751
And you say, "What direction should I walk

00:03:13.751 --> 00:03:15.760
"to increase my altitude the fastest?"

00:03:15.760 --> 00:03:17.896
You want to get uphill
as quickly as possible.

00:03:17.896 --> 00:03:19.022
And from that point,

00:03:19.022 --> 00:03:22.551
you might walk what looks
like straight up there.

00:03:22.551 --> 00:03:23.704
You certainly wouldn't go around,

00:03:23.704 --> 00:03:25.388
and this way you wouldn't go down.

00:03:25.388 --> 00:03:27.571
So you might go straight up there.

00:03:27.571 --> 00:03:31.088
And if you project your point
down onto the input space,

00:03:31.088 --> 00:03:34.223
so this is the point above which you are,

00:03:34.223 --> 00:03:37.462
that vector, the one
that's gonna get you going

00:03:37.462 --> 00:03:40.191
uphill the fastest, the
direction you should walk.

00:03:40.191 --> 00:03:41.688
For this graph, it should
kind of make sense,

00:03:41.688 --> 00:03:43.685
is directly away from the origin,

00:03:43.685 --> 00:03:45.090
'cause here, I'll erase this

00:03:45.090 --> 00:03:48.046
'cause once I start moving
things, that won't stick.

00:03:48.046 --> 00:03:51.529
If you were to look at
things from the very bottom,

00:03:51.529 --> 00:03:53.967
any point that you are on the
mountain on the graph here

00:03:53.967 --> 00:03:55.987
and when you want to increase the fastest,

00:03:55.987 --> 00:03:57.984
you should just go directly
away from the origin

00:03:57.984 --> 00:03:59.725
'cause that's when it's the steepest.

00:03:59.725 --> 00:04:01.130
And all of these vectors are also pointing

00:04:01.130 --> 00:04:02.872
directly away from the origin.

00:04:02.872 --> 00:04:05.612
So people will say the gradient
points in the direction

00:04:05.612 --> 00:04:10.413
of steepest ascent, that might
even be worth writing down.

00:04:13.390 --> 00:04:15.916
Direction of steepest ascent.

00:04:18.963 --> 00:04:20.031
And let's just see what that looks like

00:04:20.031 --> 00:04:21.889
in the context of another example.

00:04:21.889 --> 00:04:24.652
So I'll pull up another graph here,

00:04:24.652 --> 00:04:27.044
pull up another graph
and its vector field.

00:04:27.044 --> 00:04:28.971
So this graph, it's all negative values,

00:04:28.971 --> 00:04:30.608
it's all below the xy-plane,

00:04:30.608 --> 00:04:32.907
and it's got these two different peaks.

00:04:32.907 --> 00:04:34.567
And I've also drawn the gradient field,

00:04:34.567 --> 00:04:36.141
which is the word for the vector field

00:04:36.141 --> 00:04:38.800
representing the gradient on top.

00:04:38.800 --> 00:04:41.424
And you'll notice near the peak

00:04:41.424 --> 00:04:43.850
all of the vectors are pointing

00:04:43.850 --> 00:04:45.267
kind of in the uphill direction,

00:04:45.267 --> 00:04:48.100
sort of telling you to go
towards that peak in some way.

00:04:48.100 --> 00:04:51.188
And as you get a feel
around, you can see here,

00:04:51.188 --> 00:04:53.905
this very top one, like the
point that it's stemming from

00:04:53.905 --> 00:04:56.470
corresponds with something
just a little bit shy

00:04:56.470 --> 00:04:57.864
of the peak there.

00:04:57.864 --> 00:05:00.302
And everybody's telling you to go uphill.

00:05:00.302 --> 00:05:02.508
Each vector is telling
you which way to walk

00:05:02.508 --> 00:05:05.004
to increase the altitude
on the graph the fastest.

00:05:05.004 --> 00:05:07.268
It's the direction of steepest ascent.

00:05:07.268 --> 00:05:08.731
And that's what the direction means,

00:05:08.731 --> 00:05:11.041
but what does the length mean?

00:05:11.041 --> 00:05:11.589
Well, if you take a look,

00:05:11.589 --> 00:05:13.609
take a look at these red vectors here.

00:05:13.609 --> 00:05:16.441
So red means that they should
be considered very, very long.

00:05:16.441 --> 00:05:18.490
And the graph itself,

00:05:18.490 --> 00:05:19.901
the point they correspond to on the graph

00:05:19.901 --> 00:05:21.247
is just way off screen for us

00:05:21.247 --> 00:05:22.629
because this graph gets really steep

00:05:22.629 --> 00:05:24.765
and really negative very fast.

00:05:24.765 --> 00:05:26.367
So the points these correspond to

00:05:26.367 --> 00:05:28.364
have really, really steep slopes

00:05:28.364 --> 00:05:29.892
whereas these blue ones over here,

00:05:29.892 --> 00:05:31.738
you know, it's kind of a
relatively shallow slope.

00:05:31.738 --> 00:05:33.421
By the time you're getting to the peak,

00:05:33.421 --> 00:05:35.082
things start leveling off.

00:05:35.082 --> 00:05:37.113
So the length of the gradient vector

00:05:37.113 --> 00:05:38.599
actually tells you the steepness

00:05:38.599 --> 00:05:40.840
of that direction of steepest ascent.

00:05:40.840 --> 00:05:43.351
But one thing I want to point out here,

00:05:43.351 --> 00:05:46.343
it doesn't really make sense
immediately looking at it,

00:05:46.343 --> 00:05:49.415
why just throwing the partial
derivatives into a vector

00:05:49.415 --> 00:05:53.106
is gonna give you this
direction of steepest ascent.

00:05:53.106 --> 00:05:54.152
Ultimately it will.

00:05:54.152 --> 00:05:55.064
We're gonna talk through that

00:05:55.064 --> 00:05:57.583
and I hope to make that
connection pretty clear,

00:05:57.583 --> 00:06:00.346
but unless you're some
kind of intuitive genius,

00:06:00.346 --> 00:06:03.806
I don't think that connection
is at all obvious at first.

00:06:03.806 --> 00:06:05.281
But you will see it in due time.

00:06:05.281 --> 00:06:06.100
It's gonna require something

00:06:06.100 --> 00:06:08.620
called the directional derivative.

00:06:08.620 --> 00:06:10.173
See you next video.


Why the gradient is the direction of steepest ascent

WEBVTT
Kind: captions
Language: en

00:00:00.336 --> 00:00:01.853
- [Voiceover] So far,
when I've talked about

00:00:01.853 --> 00:00:03.457
the gradient of a function,

00:00:03.457 --> 00:00:05.199
and let's think about this as a

00:00:05.199 --> 00:00:07.275
multi-variable function
with just two inputs.

00:00:07.275 --> 00:00:09.272
Those are the easiest to think about.

00:00:09.272 --> 00:00:12.569
So maybe it's something like
x squared plus y squared,

00:00:12.569 --> 00:00:14.652
a very friendly function.

00:00:16.003 --> 00:00:17.070
When I've talked about the gradient,

00:00:17.070 --> 00:00:18.951
I've left open a mystery.

00:00:18.951 --> 00:00:20.693
We have the way of computing it,

00:00:20.693 --> 00:00:22.771
and the way that you
think about computing it

00:00:22.771 --> 00:00:23.828
is you just take this vector,

00:00:23.828 --> 00:00:26.300
and you just throw the
partial derivatives in there.

00:00:26.300 --> 00:00:28.530
Partial with respect to x,

00:00:28.530 --> 00:00:31.363
and the partial with respect to y,

00:00:32.986 --> 00:00:35.039
and if it was a higher dimensional input,

00:00:35.039 --> 00:00:37.999
then the output would have as
many variables as you need.

00:00:37.999 --> 00:00:39.590
If it was f of x,y,z,

00:00:39.590 --> 00:00:42.678
you'd have partial x,
partial y, partial z.

00:00:42.678 --> 00:00:44.153
And this is the way to compute it.

00:00:44.153 --> 00:00:46.184
But then I gave you a graphical intuition.

00:00:46.184 --> 00:00:48.994
I said that it points in the
direction of steepest ascent,

00:00:48.994 --> 00:00:50.321
and maybe the way you think about that

00:00:50.321 --> 00:00:52.690
is you have your input space,

00:00:52.690 --> 00:00:55.368
which in this case is the x,y plane,

00:00:55.368 --> 00:00:58.630
and you think of it as
somehow mapping over

00:00:58.630 --> 00:01:00.128
to the number line,

00:01:00.128 --> 00:01:01.556
to your output space,

00:01:01.556 --> 00:01:03.901
and if you have a given point somewhere,

00:01:03.901 --> 00:01:04.795
the question is,

00:01:04.795 --> 00:01:06.142
of all the possible directions

00:01:06.142 --> 00:01:08.290
that you can move away from this point,

00:01:08.290 --> 00:01:10.368
all those different
directions you could go,

00:01:10.368 --> 00:01:12.470
which one of them--

00:01:12.470 --> 00:01:13.898
this point will land somewhere

00:01:13.898 --> 00:01:14.885
on the function,

00:01:14.885 --> 00:01:16.383
and as you move in the various directions

00:01:16.383 --> 00:01:18.669
maybe one of them nudges
your output a little bit,

00:01:18.669 --> 00:01:20.063
one of them nudges it a lot,

00:01:20.063 --> 00:01:21.670
one of it slides it negative,

00:01:21.670 --> 00:01:23.395
one of them slides it negative a lot.

00:01:23.395 --> 00:01:25.079
Which one of these directions

00:01:25.079 --> 00:01:28.539
results in the greatest
increase to your function?

00:01:28.539 --> 00:01:29.665
And this was the loose intuition.

00:01:29.665 --> 00:01:31.383
If you want to think in terms of graphs,

00:01:31.383 --> 00:01:34.030
we could look over at the
graph of f of x squared,

00:01:34.030 --> 00:01:35.609
and this is the gradient field.

00:01:35.609 --> 00:01:37.815
All of these vectors in the x,y plane

00:01:37.815 --> 00:01:38.872
are the gradients.

00:01:38.872 --> 00:01:40.369
As you kind of look from below,

00:01:40.369 --> 00:01:41.670
you can maybe see why

00:01:41.670 --> 00:01:44.154
each one of these points in
the direction you should move

00:01:44.154 --> 00:01:46.337
to walk uphill on that graph

00:01:46.337 --> 00:01:47.800
as fast as you can.

00:01:47.800 --> 00:01:48.821
If you're a mountain climber,

00:01:48.821 --> 00:01:51.016
and you want to get to the
top as quickly as possible,

00:01:51.016 --> 00:01:53.338
these tell you the direction
that you should move

00:01:53.338 --> 00:01:54.232
to go as quickly.

00:01:54.232 --> 00:01:57.517
This is why you call it
direction of steepest ascent.

00:01:57.517 --> 00:01:59.017
So back over here,

00:02:00.289 --> 00:02:01.659
I don't see the connection immediately,

00:02:01.659 --> 00:02:03.324
or at least when I was
first learning about it,

00:02:03.324 --> 00:02:04.417
it wasn't clear

00:02:04.417 --> 00:02:08.121
why this combination
of partial derivatives

00:02:08.121 --> 00:02:11.163
has anything to do with
choosing the best direction.

00:02:11.163 --> 00:02:13.845
And now that we've learned about
the directional derivative,

00:02:13.845 --> 00:02:16.104
I can give you a little
bit of an intuition.

00:02:16.104 --> 00:02:17.989
So let's say instead of thinking about

00:02:17.989 --> 00:02:19.951
all the possible directions,

00:02:19.951 --> 00:02:21.875
and all of the possible changes

00:02:21.875 --> 00:02:24.027
to the output that they have,

00:02:24.027 --> 00:02:26.527
so I'll fill in my line there.

00:02:27.664 --> 00:02:28.694
Let's say you just have,

00:02:28.694 --> 00:02:29.704
you've got your point

00:02:29.704 --> 00:02:31.260
where you're evaluating things,

00:02:31.260 --> 00:02:33.907
and then you just have a single vector,

00:02:33.907 --> 00:02:35.195
and let's actually make it a unit vector.

00:02:35.195 --> 00:02:36.995
Let's make it the case

00:02:36.995 --> 00:02:38.620
that this guy has a length of one.

00:02:38.620 --> 00:02:39.549
So I'll go over here, and I'll just

00:02:39.549 --> 00:02:41.105
think of that guy as being V,

00:02:41.105 --> 00:02:44.022
and say that V has a length of one,

00:02:45.168 --> 00:02:47.723
so this is our vector.

00:02:47.723 --> 00:02:48.556
We know now,

00:02:48.556 --> 00:02:50.555
having learned about the
directional derivative,

00:02:50.555 --> 00:02:51.809
that you can tell

00:02:51.809 --> 00:02:53.678
the rate at which the function changes

00:02:53.678 --> 00:02:55.548
as you move in this direction

00:02:55.548 --> 00:02:59.158
by taking a directional derivative

00:02:59.158 --> 00:02:59.991
of your function,

00:02:59.991 --> 00:03:00.824
and let's say this point,

00:03:00.824 --> 00:03:02.746
I don't know, what's a
good name for this point?

00:03:02.746 --> 00:03:03.996
Just like, a,b.

00:03:04.940 --> 00:03:06.440
a,b is this point.

00:03:08.028 --> 00:03:10.528
When you evaluate this at a,b,

00:03:12.312 --> 00:03:13.905
and the way that you do that

00:03:13.905 --> 00:03:16.738
is just dotting the gradient of f.

00:03:18.461 --> 00:03:21.545
I should say dotting it,
evaluate it at that point,

00:03:21.545 --> 00:03:23.704
'cause gradient is a
vector valued function,

00:03:23.704 --> 00:03:25.817
and we just want a specific vector here,

00:03:25.817 --> 00:03:28.650
so, evaluating that at your point,

00:03:29.867 --> 00:03:30.700
a,b,

00:03:33.491 --> 00:03:35.616
together with whatever the vector is,

00:03:35.616 --> 00:03:37.810
whatever that value is,

00:03:37.810 --> 00:03:41.577
and in this case we're
thinking of V as a unit vector.

00:03:41.577 --> 00:03:42.410
So this,

00:03:44.787 --> 00:03:47.399
this is how you tell the rate of change,

00:03:47.399 --> 00:03:49.593
and when I originally introduced
the directional derivative,

00:03:49.593 --> 00:03:52.879
I gave kind of an indication why.

00:03:52.879 --> 00:03:55.398
If you imagine dotting this together with,

00:03:55.398 --> 00:03:58.458
let's say it was a vector
that's like one two,

00:03:58.458 --> 00:03:59.690
really you're thinking
this vector represents

00:03:59.690 --> 00:04:01.153
one step in the x direction,

00:04:01.153 --> 00:04:02.743
two steps in the y direction,

00:04:02.743 --> 00:04:04.775
so the amount that it
changes things should be

00:04:04.775 --> 00:04:06.183
one times the change caused by

00:04:06.183 --> 00:04:07.863
a pure step in the x direction,

00:04:07.863 --> 00:04:10.059
plus two times a change caused by

00:04:10.059 --> 00:04:11.474
a pure step in the y direction.

00:04:11.474 --> 00:04:12.856
So that was kind of the loose intuition.

00:04:12.856 --> 00:04:14.853
You can see the directional
derivative video

00:04:14.853 --> 00:04:18.951
if you want a little bit
more discussion on that.

00:04:18.951 --> 00:04:20.623
And this is the formula that you have.

00:04:20.623 --> 00:04:22.359
But this starts to give us the key for

00:04:22.359 --> 00:04:25.371
how we could choose the
direction of steepest ascent,

00:04:25.371 --> 00:04:27.124
'cause now, what we're really asking,

00:04:27.124 --> 00:04:28.947
when we say which one of
these changes things the most,

00:04:28.947 --> 00:04:30.863
maybe when you move in that direction

00:04:30.863 --> 00:04:34.067
it changes f a little bit negatively,

00:04:34.067 --> 00:04:36.209
and we want to know,

00:04:36.209 --> 00:04:38.042
does another vector W,

00:04:38.900 --> 00:04:40.769
is the change caused by
that gonna be positive?

00:04:40.769 --> 00:04:42.731
Is it gonna be as big as possible?

00:04:42.731 --> 00:04:45.216
What we're doing is we're saying

00:04:45.216 --> 00:04:47.076
find the maximum

00:04:47.076 --> 00:04:48.713
for all unit vectors,

00:04:48.713 --> 00:04:51.035
so for all vectors V

00:04:51.035 --> 00:04:53.798
that satisfy the property
that their length is one,

00:04:53.798 --> 00:04:56.898
find the maximum of the dot product

00:04:56.898 --> 00:04:59.289
between f evaluated at that point,

00:04:59.289 --> 00:05:03.063
evaluated at whatever point we care about,

00:05:03.063 --> 00:05:03.896
and V.

00:05:05.222 --> 00:05:06.722
Find that maximum.

00:05:08.833 --> 00:05:10.632
Well, let's just think about

00:05:10.632 --> 00:05:12.780
what the dot product represents.

00:05:12.780 --> 00:05:13.802
So let's say we go over here,

00:05:13.802 --> 00:05:16.855
and let's say we evaluate
the gradient vector

00:05:16.855 --> 00:05:18.190
and it turns out that the gradient

00:05:18.190 --> 00:05:19.537
points in this direction,

00:05:19.537 --> 00:05:21.604
and maybe, it doesn't
have to be a unit vector,

00:05:21.604 --> 00:05:24.901
it might be something very long like that.

00:05:24.901 --> 00:05:27.420
So if you imagine some vector V,

00:05:27.420 --> 00:05:29.046
some unit vector V,

00:05:29.046 --> 00:05:32.006
let's say it was taking
off in this direction.

00:05:32.006 --> 00:05:34.363
The way that you interpret
this dot product,

00:05:34.363 --> 00:05:36.871
the dot product between the gradient f

00:05:36.871 --> 00:05:38.534
and this new vector V,

00:05:38.534 --> 00:05:41.062
is you would project that vector

00:05:41.062 --> 00:05:43.698
directly, kind of a
perpendicular projection

00:05:43.698 --> 00:05:45.207
onto your gradient vector,

00:05:45.207 --> 00:05:47.204
and you'd say what's that length?

00:05:47.204 --> 00:05:48.539
What's that length right there?

00:05:48.539 --> 00:05:49.897
And just as an example,
it would be something

00:05:49.897 --> 00:05:51.244
a little bit less than one, right?

00:05:51.244 --> 00:05:52.138
'Cause this is a unit vector.

00:05:52.138 --> 00:05:55.555
So as an example, let's say that was 0.7.

00:05:56.445 --> 00:05:58.472
And then you'd multiply that by the length

00:05:58.472 --> 00:05:59.708
of the gradient itself,

00:05:59.708 --> 00:06:01.937
of the vector against
which you're dotting,

00:06:01.937 --> 00:06:04.491
and maybe that guy, maybe the length

00:06:04.491 --> 00:06:06.616
of the entire gradient vector,

00:06:06.616 --> 00:06:08.729
just, again, as an
example, maybe that's two.

00:06:08.729 --> 00:06:11.352
It doesn't have to be,
it could be anything.

00:06:11.352 --> 00:06:13.881
But the way that you interpret
this whole dot product then

00:06:13.881 --> 00:06:15.866
is to take the product of those two.

00:06:15.866 --> 00:06:17.449
You would take 0.7,

00:06:19.036 --> 00:06:20.742
the length of your projection,

00:06:20.742 --> 00:06:23.841
times the length of the original vector.

00:06:23.841 --> 00:06:25.700
And the question is

00:06:25.700 --> 00:06:26.838
when is this maximized?

00:06:26.838 --> 00:06:28.951
What unit vector maximizes this?

00:06:28.951 --> 00:06:30.135
And if you start to imagine maybe

00:06:30.135 --> 00:06:32.085
swinging that unit vector around,

00:06:32.085 --> 00:06:34.082
so if, instead of that guy,

00:06:34.082 --> 00:06:35.742
you were to use

00:06:35.742 --> 00:06:39.121
one that pointed a little bit more

00:06:39.121 --> 00:06:40.456
closely in the direction,

00:06:40.456 --> 00:06:41.431
then it's projection

00:06:41.431 --> 00:06:42.360
would be a little bit longer.

00:06:42.360 --> 00:06:45.530
Maybe that projection would
be like 0.75 or something.

00:06:45.530 --> 00:06:47.004
If you take the unit vector

00:06:47.004 --> 00:06:49.930
that points directly in the same direction

00:06:49.930 --> 00:06:52.287
as that full vector,

00:06:52.287 --> 00:06:53.297
then the length of its projection

00:06:53.297 --> 00:06:55.131
is just the length of the vector itself.

00:06:55.131 --> 00:06:56.431
It would be one,

00:06:56.431 --> 00:06:59.520
because projecting it doesn't
change what it is at all.

00:06:59.520 --> 00:07:01.064
So it shouldn't be too
hard to convince yourself,

00:07:01.064 --> 00:07:04.361
and if you have shaky
intuitions on the dot product,

00:07:04.361 --> 00:07:06.184
I'd suggest finding the videos we have

00:07:06.184 --> 00:07:07.449
on Khan Academy for those.

00:07:07.449 --> 00:07:11.449
Sal does a great job
giving that deep intuition.

00:07:12.314 --> 00:07:13.707
It should kind of make sense why

00:07:13.707 --> 00:07:16.714
the unit vector that points
in the same direction

00:07:16.714 --> 00:07:19.396
as your gradient is gonna
be what maximizes it,

00:07:19.396 --> 00:07:20.406
so the answer here,

00:07:20.406 --> 00:07:22.809
the answer to what vector maximizes this

00:07:22.809 --> 00:07:25.932
is gonna be, well, it's
the gradient itself, right?

00:07:25.932 --> 00:07:27.593
It is that gradient vector

00:07:27.593 --> 00:07:29.543
evaluated at the point we care about,

00:07:29.543 --> 00:07:30.925
except you'd normalize it, right?

00:07:30.925 --> 00:07:32.956
Because we're only
considering unit vectors,

00:07:32.956 --> 00:07:34.210
so to do that, you just

00:07:34.210 --> 00:07:36.335
divide it by whatever it's magnitude is.

00:07:36.335 --> 00:07:38.378
If its magnitude was
already one, it stays one.

00:07:38.378 --> 00:07:39.876
If its magnitude was two,

00:07:39.876 --> 00:07:41.629
you're dividing it down

00:07:41.629 --> 00:07:42.686
by a half.

00:07:42.686 --> 00:07:43.777
So this is your answer.

00:07:43.777 --> 00:07:46.134
This is the direction of steepest ascent.

00:07:46.134 --> 00:07:48.374
So, I think one thing to notice here

00:07:48.374 --> 00:07:50.685
is the most fundamental fact

00:07:50.685 --> 00:07:52.473
is that the gradient is this tool

00:07:52.473 --> 00:07:54.423
for computing directional derivatives.

00:07:54.423 --> 00:07:55.818
You can think of that vector as something

00:07:55.818 --> 00:07:58.521
that you really want to dot against,

00:07:58.521 --> 00:08:00.983
and that's actually a
pretty powerful thought,

00:08:00.983 --> 00:08:02.399
is that the gradient,
it's not just a vector,

00:08:02.399 --> 00:08:03.943
it's a vector that loves to be dotted

00:08:03.943 --> 00:08:05.197
together with other things.

00:08:05.197 --> 00:08:06.509
That's the fundamental.

00:08:06.509 --> 00:08:09.551
And as a consequence of that,

00:08:09.551 --> 00:08:11.502
the direction of steepest ascent

00:08:11.502 --> 00:08:13.777
is that vector itself

00:08:13.777 --> 00:08:15.739
because anything, if you're saying

00:08:15.739 --> 00:08:18.212
what maximizes the dot
product with that thing,

00:08:18.212 --> 00:08:19.547
it's, well, the vector that points in

00:08:19.547 --> 00:08:21.544
the same direction as that thing.

00:08:21.544 --> 00:08:23.785
And this can also give
us an interpretation

00:08:23.785 --> 00:08:27.163
for the length of the gradient.

00:08:27.163 --> 00:08:28.173
We know the direction is the

00:08:28.173 --> 00:08:29.311
direction of steepest ascent,

00:08:29.311 --> 00:08:31.598
but what is the length mean?

00:08:31.598 --> 00:08:32.736
So, let's give this guy a name.

00:08:32.736 --> 00:08:35.128
Let's give this normalized
version of it a name.

00:08:35.128 --> 00:08:37.485
I'm just gonna call it W.

00:08:37.485 --> 00:08:38.959
So W will be the unit vector

00:08:38.959 --> 00:08:41.931
that points in the
direction of the gradient.

00:08:41.931 --> 00:08:44.497
If you take the directional derivative

00:08:44.497 --> 00:08:47.469
in the direction of W of f,

00:08:47.469 --> 00:08:50.883
what that means is the gradient of f

00:08:50.883 --> 00:08:52.466
dotted with that W.

00:08:53.402 --> 00:08:55.143
And if you kind of spell out

00:08:55.143 --> 00:08:56.643
what W means here,

00:08:58.313 --> 00:09:01.169
that means you're taking
the gradient of the vector

00:09:01.169 --> 00:09:03.363
dotted with itself,

00:09:03.363 --> 00:09:05.639
but because it's W and not the gradient,

00:09:05.639 --> 00:09:07.508
we're normalizing.

00:09:07.508 --> 00:09:09.795
We're dividing that,
not by magnitude of f,

00:09:09.795 --> 00:09:11.014
that doesn't really make sense,

00:09:11.014 --> 00:09:12.396
but by the value of the gradient,

00:09:12.396 --> 00:09:13.696
and all of these,

00:09:13.696 --> 00:09:15.043
I'm just writing gradient of f,

00:09:15.043 --> 00:09:16.750
but maybe you should be thinking about

00:09:16.750 --> 00:09:19.333
gradient of f evaluated at a,b,

00:09:20.604 --> 00:09:21.823
but I'm just being kind of lazy,

00:09:21.823 --> 00:09:24.146
and just writing gradient of f.

00:09:24.146 --> 00:09:27.199
And the top, when you take
the dot product with itself,

00:09:27.199 --> 00:09:29.149
what that means is

00:09:29.149 --> 00:09:31.482
the square of its magnitude.

00:09:34.850 --> 00:09:38.850
But the whole thing is
divided by the magnitude.

00:09:40.376 --> 00:09:41.479
So you can kind of cancel that out.

00:09:41.479 --> 00:09:44.323
You could say this
doesn't need to be there,

00:09:44.323 --> 00:09:46.320
that exponent doesn't need to be there,

00:09:46.320 --> 00:09:49.897
and basically, the directional derivative

00:09:49.897 --> 00:09:52.167
in the direction of the gradient itself

00:09:52.167 --> 00:09:55.499
has a value equal to the
magnitude of the gradient.

00:09:55.499 --> 00:09:58.889
So this tells you when you're
moving in that direction,

00:09:58.889 --> 00:10:00.305
in the direction of the gradient,

00:10:00.305 --> 00:10:02.140
the rate at which the function changes

00:10:02.140 --> 00:10:04.671
is given by the magnitude of the gradient.

00:10:04.671 --> 00:10:06.296
So it's this really magical vector.

00:10:06.296 --> 00:10:07.129
It does a lot of things.

00:10:07.129 --> 00:10:10.000
It's the tool that lets you
dot against other vectors

00:10:10.000 --> 00:10:11.706
to tell you the directional derivative.

00:10:11.706 --> 00:10:14.307
As a consequence, it's the
direction of steepest ascent,

00:10:14.307 --> 00:10:16.466
and its magnitude tells
you the rate at which

00:10:16.466 --> 00:10:18.336
things change while you're moving in that

00:10:18.336 --> 00:10:19.764
direction of steepest ascent.

00:10:19.764 --> 00:10:21.264
It's just really a

00:10:22.295 --> 00:10:25.418
core part of scalar valued
multi-variable functions,

00:10:25.418 --> 00:10:28.181
and it is the extension of the derivative

00:10:28.181 --> 00:10:31.625
in every sense that you could
want a derivative to extend.


Directional derivative

WEBVTT
Kind: captions
Language: en

00:00:00.575 --> 00:00:01.507
- [Voiceover] Hello everyone.

00:00:01.507 --> 00:00:04.107
So here I'm gonna talk about
the directional derivative

00:00:04.107 --> 00:00:05.474
and that's a way to extend the idea

00:00:05.474 --> 00:00:07.174
of a partial derivative.

00:00:07.174 --> 00:00:09.240
And partial derivatives, if you remember,

00:00:09.240 --> 00:00:10.374
have to do with functions

00:00:10.374 --> 00:00:12.375
with some kind of multi-variable input,

00:00:12.375 --> 00:00:13.632
and I'll just use two inputs

00:00:13.632 --> 00:00:15.112
because that's the easiest to think about,

00:00:15.112 --> 00:00:18.341
and it could be some
single variable output.

00:00:18.341 --> 00:00:20.141
It could also deal with
vector variable outputs.

00:00:20.141 --> 00:00:21.311
We haven't gotten to that yet.

00:00:21.311 --> 00:00:23.044
So, we'll just think
about a single variable,

00:00:23.044 --> 00:00:25.074
ordinary real number output that's,

00:00:25.074 --> 00:00:27.307
you know, an expression of x and y,

00:00:27.307 --> 00:00:28.875
and the partial derivative.

00:00:28.875 --> 00:00:30.700
One of the ways I said
you could think about it

00:00:30.700 --> 00:00:33.174
is to take a look at the input space,

00:00:33.174 --> 00:00:34.308
your x and y-plane.

00:00:34.308 --> 00:00:35.475
So, this would be the x-axis,

00:00:35.475 --> 00:00:36.641
This is y.

00:00:36.641 --> 00:00:38.141
And you know, vaguely, in your mind,

00:00:38.141 --> 00:00:40.874
you're thinking that somehow
this outputs to a line.

00:00:40.874 --> 00:00:43.141
This outputs to just the real numbers.

00:00:43.141 --> 00:00:44.707
And maybe you're thinking
about a transformation

00:00:44.707 --> 00:00:46.474
that takes it there, or maybe you're just

00:00:46.474 --> 00:00:48.141
thinking, "Okay, this is the input space.

00:00:48.141 --> 00:00:49.541
"That's the output."

00:00:49.541 --> 00:00:50.908
And when you take the partial derivative,

00:00:50.908 --> 00:00:52.441
at some kind of point...

00:00:52.441 --> 00:00:53.841
So, I'll write it out like

00:00:53.841 --> 00:00:56.874
partial derivative of f with respect to x

00:00:56.874 --> 00:00:59.063
at a point like one, two.

00:00:59.973 --> 00:01:01.274
You think about that point.

00:01:01.274 --> 00:01:04.508
You know one, y is equal to two.

00:01:04.508 --> 00:01:06.174
And if you're taking with respect to x,

00:01:06.174 --> 00:01:07.713
you think about just
nudging it a little bit

00:01:07.713 --> 00:01:09.471
in that x-direction, and you see

00:01:09.471 --> 00:01:12.216
what the resulting nudge
is in the output space.

00:01:12.216 --> 00:01:13.441
And the ratio between the size

00:01:13.441 --> 00:01:15.374
of that resulting nudge, the original one,

00:01:15.374 --> 00:01:17.041
the ratio between, you know, partial-f

00:01:17.041 --> 00:01:19.774
and partial-x is the value that you want.

00:01:19.774 --> 00:01:21.408
And when you did it with respect to y,

00:01:21.408 --> 00:01:23.576
you know you were thinking about traveling

00:01:23.576 --> 00:01:26.608
in a different direction,
maybe you nudge it straight up.

00:01:26.608 --> 00:01:27.707
And you're wondering, "Okay, how does

00:01:27.707 --> 00:01:29.274
"that influence the output?"

00:01:29.274 --> 00:01:31.874
And the question here, with
directional derivatives,

00:01:31.874 --> 00:01:34.273
what if you have some vector, v,

00:01:34.273 --> 00:01:37.341
I'll give a little
vector hat on top of it,

00:01:37.341 --> 00:01:39.675
that, you know, I don't
know, let's say it's

00:01:39.675 --> 00:01:43.881
negative one, two, is the vector.

00:01:44.741 --> 00:01:46.076
So you'd be thinking about that as a step

00:01:46.076 --> 00:01:47.775
of negative one in the x-direction,

00:01:47.775 --> 00:01:49.740
and then two more in the y-direction.

00:01:49.740 --> 00:01:53.408
So, it's gonna be something
that ends up there.

00:01:53.408 --> 00:01:54.674
This is your vector, v.

00:01:54.674 --> 00:01:56.208
At least, if you're
thinking of v as stemming

00:01:56.208 --> 00:01:57.575
from the original point.

00:01:57.575 --> 00:01:58.941
And you're wondering, "What does a nudge

00:01:58.941 --> 00:02:01.341
"in that direction do
to the function itself?"

00:02:01.341 --> 00:02:03.541
And remember, with these
original, you know, nudges

00:02:03.541 --> 00:02:05.208
in the x-direction, nudges in the y,

00:02:05.208 --> 00:02:06.608
you're not really thinking of it as,

00:02:06.608 --> 00:02:07.941
"You know, this is kind of a large step."

00:02:07.941 --> 00:02:09.207
You're really thinking
of it as something as

00:02:09.207 --> 00:02:10.607
itty-itty-bitty-bitty.

00:02:10.607 --> 00:02:11.940
You know it's not that,
but it's really something

00:02:11.940 --> 00:02:13.875
very, very small, and
formally you'd be thinking

00:02:13.875 --> 00:02:16.208
about the limit as this gets
really, really. really small

00:02:16.208 --> 00:02:17.374
approaching zero.

00:02:17.374 --> 00:02:19.408
And this gets really, really
small approaching zero,

00:02:19.408 --> 00:02:21.308
what does the ration of the two approach?

00:02:21.308 --> 00:02:23.208
And similarly with the
y, you're not thinking

00:02:23.208 --> 00:02:25.478
of it as something,
"This is pretty sizable,"

00:02:25.478 --> 00:02:27.541
but it would be something
really, really small.

00:02:27.541 --> 00:02:29.041
And the directional derivative is similar.

00:02:29.041 --> 00:02:30.674
You're not thinking of the actual vector

00:02:30.674 --> 00:02:32.495
actually taking a step along that,

00:02:32.495 --> 00:02:34.774
but you'd be thinking
of taking a step along,

00:02:34.774 --> 00:02:38.931
say, h multiplied by that vector,

00:02:38.931 --> 00:02:41.441
and h might represent some
really, really small numbers.

00:02:41.441 --> 00:02:46.155
You know, maybe this here is like 0.001.

00:02:46.155 --> 00:02:47.441
And when you're doing this formula,

00:02:47.441 --> 00:02:50.171
you'd just be thinking the
limit as h goes to zero.

00:02:50.171 --> 00:02:51.808
So, the directional derivative is saying

00:02:51.808 --> 00:02:54.241
when you take a slight
nudge in the direction

00:02:54.241 --> 00:02:57.907
of that vector, what is the
resulting change to the output?

00:02:57.907 --> 00:02:59.341
And one way to think
about this is you say,

00:02:59.341 --> 00:03:01.708
"Well, that slight nudge of the vector..."

00:03:01.708 --> 00:03:03.240
If we actually expand things out

00:03:03.240 --> 00:03:04.681
and we look at the definition itself,

00:03:04.681 --> 00:03:07.537
it'll be negative h, negative
one times that component,

00:03:07.537 --> 00:03:10.017
and then two-h here.

00:03:10.017 --> 00:03:14.475
So it's kind of like you
took negative one nudge,

00:03:14.475 --> 00:03:17.841
in the x-direction, and then
two nudges in the y-direction.

00:03:17.841 --> 00:03:20.841
You know, so for whatever
your nudge in the v-direction,

00:03:20.841 --> 00:03:23.841
there, you take a negative one step by x,

00:03:23.841 --> 00:03:25.941
and then two of them up by y.

00:03:25.941 --> 00:03:27.378
So, when we actually write this out...

00:03:27.378 --> 00:03:29.337
The notation, by the way, is

00:03:29.337 --> 00:03:32.475
you take that same nabla from the gradient

00:03:32.475 --> 00:03:34.175
but then you put the vector down here.

00:03:34.175 --> 00:03:35.675
So, this is the directional derivative

00:03:35.675 --> 00:03:36.907
in the direction of v.

00:03:36.907 --> 00:03:39.185
And there's a whole bunch
of other notations too.

00:03:39.185 --> 00:03:41.908
You know, I think there's
like derivative of f

00:03:41.908 --> 00:03:43.807
with respect to that vector,

00:03:43.807 --> 00:03:45.174
is one way people think about it.

00:03:45.174 --> 00:03:46.773
Some people will just write like partial

00:03:46.773 --> 00:03:48.274
with a little subscript vector.

00:03:48.274 --> 00:03:49.674
There's a whole bunch
of different notations,

00:03:49.674 --> 00:03:50.708
but this is the one I like.

00:03:50.708 --> 00:03:54.013
You think that nabla with
the little f down there

00:03:54.013 --> 00:03:56.707
with a little v for your vector, of f,

00:03:56.707 --> 00:03:58.897
and it's still a function of x and y.

00:04:01.607 --> 00:04:02.540
And the reason I like this is

00:04:02.540 --> 00:04:05.074
it's indicative of how
you end up calculating it,

00:04:05.074 --> 00:04:06.975
which I'll talk about
at the end of the video.

00:04:06.975 --> 00:04:09.041
And for this particular example,

00:04:09.041 --> 00:04:11.241
a good guess that you
might have is to say,

00:04:11.241 --> 00:04:13.908
"Well, we take a negative
step in the x-direction."

00:04:13.908 --> 00:04:16.074
So you think of it as whatever the change

00:04:16.074 --> 00:04:18.274
that's caused by such a
step in the x-direction,

00:04:18.274 --> 00:04:19.641
you do the negative of that,

00:04:19.641 --> 00:04:21.674
and then it's two steps
in the y-direction.

00:04:21.674 --> 00:04:23.375
So, whatever the change
caused by a tiny step

00:04:23.375 --> 00:04:25.010
in the y-direction,

00:04:26.510 --> 00:04:28.775
let's just take two of those.

00:04:28.775 --> 00:04:32.675
Two times partial-f, partial-y.

00:04:32.675 --> 00:04:35.773
And this is actually how you calculate it.

00:04:35.773 --> 00:04:37.640
And if I was gonna be more general,

00:04:37.640 --> 00:04:40.241
you know, let's say we've got a vector, w.

00:04:40.241 --> 00:04:41.608
I'm going to keep it abstract

00:04:41.608 --> 00:04:46.208
and just call it a and
b, as its components,

00:04:46.208 --> 00:04:47.674
other than the specific numbers.

00:04:47.674 --> 00:04:50.311
You would say that the
directional derivative

00:04:50.311 --> 00:04:52.982
in the direction of w, whatever that is,

00:04:52.982 --> 00:04:56.407
of f is equal to

00:04:56.407 --> 00:05:01.282
a times the partial derivative
of f with respect to x

00:05:02.961 --> 00:05:05.941
plus b times the partial derivative of f,

00:05:05.941 --> 00:05:07.568
with respect to y.

00:05:09.108 --> 00:05:09.907
And this is it.

00:05:09.907 --> 00:05:12.221
This is the formula that you would use

00:05:13.441 --> 00:05:14.774
for the directional derivative.

00:05:14.774 --> 00:05:16.341
And again, the way that
you're thinking about this

00:05:16.341 --> 00:05:17.842
is you're really saying, you know,

00:05:17.842 --> 00:05:20.478
you take a little nudge
that's a in the x-direction

00:05:20.478 --> 00:05:22.107
and b in the y-direction.

00:05:22.107 --> 00:05:23.575
So, this should kind of make sense.

00:05:23.575 --> 00:05:25.241
And sometimes you see this written

00:05:25.241 --> 00:05:28.241
not with respect to the
partial derivatives themselves

00:05:28.241 --> 00:05:30.041
and the actual components, a and b,

00:05:30.041 --> 00:05:32.507
but with respect to the gradient.

00:05:32.507 --> 00:05:34.241
And this is because it
makes it much more compact,

00:05:34.241 --> 00:05:36.848
more general, if you're
dealing with other dimensions.

00:05:36.848 --> 00:05:38.568
So, we'll just write it over here.

00:05:38.568 --> 00:05:42.374
If you look at this expression,
it looks like a dot product.

00:05:42.374 --> 00:05:46.464
If you would take the dot
product of the vectors, a, b,

00:05:47.970 --> 00:05:50.540
and the one that has the
partial derivatives in it.

00:05:50.540 --> 00:05:52.541
So, what's lined up with a
is the partial derivative

00:05:52.541 --> 00:05:56.541
with respect to x, partial-f, partial-x,

00:05:56.541 --> 00:05:59.641
and what's lined up with b
is the partial derivative

00:05:59.641 --> 00:06:01.417
with respect to y.

00:06:03.677 --> 00:06:04.574
And you look at this and you say,

00:06:04.574 --> 00:06:06.841
"Hey, a, b, I mean that's
just the original vector.

00:06:06.841 --> 00:06:08.608
"Right, that's w.

00:06:08.608 --> 00:06:10.608
"That's the vector, w."

00:06:10.608 --> 00:06:12.141
And then you're dotting this with,

00:06:12.141 --> 00:06:14.241
well, partial derivative
with respect to x,

00:06:14.241 --> 00:06:16.008
in one component, the
other partial derivative

00:06:16.008 --> 00:06:17.377
in the other component.

00:06:17.377 --> 00:06:19.108
That's just the gradient.

00:06:19.108 --> 00:06:21.407
That is the gradient of f.

00:06:21.407 --> 00:06:24.397
And here, you know, it's
nabla without that little w

00:06:24.397 --> 00:06:26.384
at the bottom, and this is
why we use this notation

00:06:26.384 --> 00:06:28.314
because it's so suggestive of the way

00:06:28.314 --> 00:06:29.938
that you ultimately calculate it.

00:06:29.938 --> 00:06:33.111
So, this is really what
you'll see in a textbook,

00:06:33.111 --> 00:06:35.711
or see as the compact way of writing it.

00:06:35.711 --> 00:06:38.651
And you can see how this is
more flexible for dimensions.

00:06:38.651 --> 00:06:39.741
So, if we were talking about something

00:06:39.741 --> 00:06:41.307
that has like a five dimensional input

00:06:41.307 --> 00:06:43.508
and the vector of the
direction you move has

00:06:43.508 --> 00:06:44.542
five different components.

00:06:44.542 --> 00:06:45.375
This is flexible.

00:06:45.375 --> 00:06:46.742
When you expand it,
the gradient would have

00:06:46.742 --> 00:06:48.747
five components, and the vector itself

00:06:48.747 --> 00:06:50.475
would have five components.

00:06:50.475 --> 00:06:52.644
So, this is the directional derivative

00:06:52.644 --> 00:06:53.608
and how you calculate it.

00:06:53.608 --> 00:06:54.875
And the way you interpret,

00:06:54.875 --> 00:06:58.142
you're thinking of
moving along that vector

00:06:58.142 --> 00:07:01.075
by a tiny nudge, by a tiny,
you know, little value

00:07:01.075 --> 00:07:02.601
multiplied by that vector

00:07:02.601 --> 00:07:04.679
and saying, "How does
that change the output

00:07:04.679 --> 00:07:07.902
"and what's the ratio of
the resulting change?"

00:07:07.902 --> 00:07:09.841
And in the next video, I'll clarify that

00:07:09.841 --> 00:07:11.574
with the formal definition

00:07:11.574 --> 00:07:13.633
of the directional derivative itself.


Symmetry of second partial derivatives

WEBVTT
Kind: captions
Language: en

00:00:00.900 --> 00:00:02.153
- [Voiceover] So in
the last couple videos,

00:00:02.153 --> 00:00:03.557
I talked about partial derivatives

00:00:03.557 --> 00:00:05.047
of multivariable functions.

00:00:05.047 --> 00:00:08.125
And here, I want to talk about
second partial derivatives.

00:00:08.125 --> 00:00:10.172
So I'm gonna write some kind
of multivariable function.

00:00:10.172 --> 00:00:11.157
Let's say it's

00:00:12.447 --> 00:00:14.880
well, sine of x times y squared.

00:00:14.880 --> 00:00:17.414
Sine of x multiplied by y squared.

00:00:18.264 --> 00:00:20.124
And if you take the partial derivative,

00:00:20.124 --> 00:00:22.193
you have two options, given
that there's two variables.

00:00:22.193 --> 00:00:23.455
You can go one way

00:00:23.455 --> 00:00:25.804
and say what's the partial derivative?

00:00:25.804 --> 00:00:29.091
Partial derivative of f with respect to x.

00:00:29.091 --> 00:00:30.866
And what you do for that,

00:00:30.866 --> 00:00:32.339
x looks like a variable

00:00:32.339 --> 00:00:34.416
as far as this direction is concerned.

00:00:34.416 --> 00:00:35.714
Y looks like a constant.

00:00:35.714 --> 00:00:38.532
So we differentiate this by saying

00:00:38.532 --> 00:00:41.605
the derivative of sine of x is cosine x.

00:00:41.605 --> 00:00:44.020
You know, you're differentiating
with respect to x.

00:00:44.020 --> 00:00:46.487
And then that looks like it's
multiplied by a constant.

00:00:46.487 --> 00:00:49.649
So you just continue
multiplying that by a constant.

00:00:49.649 --> 00:00:51.207
But you could also go another direction.

00:00:51.207 --> 00:00:52.917
You could also say, you know,

00:00:52.917 --> 00:00:55.779
what's the partial
derivative with respect to y?

00:00:55.779 --> 00:00:59.160
And in that case, you're
considering y to be the variable.

00:00:59.160 --> 00:01:00.435
So here it looks at y.

00:01:00.435 --> 00:01:02.045
And says y squared looks like a variable.

00:01:02.045 --> 00:01:03.652
X looks like a constant.

00:01:03.652 --> 00:01:05.565
Sine of x then just looks
like sine of a constant,

00:01:05.565 --> 00:01:06.980
which is a constant.

00:01:06.980 --> 00:01:10.449
So that will be that constant, sine of x,

00:01:10.449 --> 00:01:12.966
multiplied by the derivative of y squared.

00:01:12.966 --> 00:01:14.747
Which is gonna be two times y.

00:01:15.897 --> 00:01:17.197
Two times y.

00:01:18.057 --> 00:01:19.681
And these are what you might call

00:01:19.681 --> 00:01:20.855
first partial derivatives.

00:01:20.855 --> 00:01:24.283
And there's some alternate
notation here, df dy.

00:01:24.283 --> 00:01:26.665
You could also say f and
then a little subscript y.

00:01:26.665 --> 00:01:28.221
And over here, similarly,

00:01:28.221 --> 00:01:31.595
you'd say f with a little subscript x.

00:01:31.595 --> 00:01:33.405
Now each of these two functions,

00:01:33.405 --> 00:01:35.127
these two partial derivatives that you get

00:01:35.127 --> 00:01:37.116
are also multivariable functions.

00:01:37.116 --> 00:01:39.197
They take in two variables
and they output a scalar.

00:01:39.197 --> 00:01:41.093
So we can do something very similar,

00:01:41.093 --> 00:01:43.118
where here you might then apply

00:01:43.118 --> 00:01:46.316
the partial derivative with respect to x

00:01:46.316 --> 00:01:49.819
to that partial derivative
of your original function

00:01:49.819 --> 00:01:52.150
with respect to x, right.

00:01:52.150 --> 00:01:55.569
It's just like a second
derivative in ordinary calculus,

00:01:55.569 --> 00:01:56.963
but this time we're doing it partial.

00:01:56.963 --> 00:01:58.611
So when you do it with respect to x,

00:01:58.611 --> 00:02:01.289
cosine x looks like cosine of a variable.

00:02:01.289 --> 00:02:04.267
The derivative of which is negative sine

00:02:04.267 --> 00:02:06.504
times that variable.

00:02:06.504 --> 00:02:09.415
And y squared here just
looks like a constant.

00:02:09.415 --> 00:02:11.648
So it just stays constant at y squared.

00:02:11.648 --> 00:02:13.820
And, similary, you could go down

00:02:13.820 --> 00:02:15.604
a different branch of options here.

00:02:15.604 --> 00:02:17.698
And say what if you did
your partial derivative

00:02:17.698 --> 00:02:19.646
with respect to y?

00:02:19.646 --> 00:02:20.866
Of that whole function,

00:02:20.866 --> 00:02:24.339
which itself is a partial
derivative with respect to x.

00:02:26.259 --> 00:02:27.714
And if you did that,

00:02:27.714 --> 00:02:31.551
then y squared now
looks like the variable.

00:02:31.551 --> 00:02:33.395
So you're gonna take
the derivative of that,

00:02:33.395 --> 00:02:34.987
which is two y.

00:02:34.987 --> 00:02:36.201
Two y.

00:02:36.201 --> 00:02:38.402
And then what's in front of
it just looks like a constant

00:02:38.402 --> 00:02:41.798
as far as the variable y is concerned.

00:02:41.798 --> 00:02:43.709
So that stays as cosine of x.

00:02:45.259 --> 00:02:46.553
And the notation here.

00:02:46.553 --> 00:02:49.107
First of all, just as in
single-variable calculus,

00:02:49.107 --> 00:02:52.407
it's common to kind of
do a abusive notation

00:02:52.407 --> 00:02:53.343
with this kind of thing

00:02:53.343 --> 00:02:56.238
and write partial squared of f

00:02:57.178 --> 00:03:01.184
divided by partial x squared.

00:03:02.454 --> 00:03:03.845
And this always, I don't know.

00:03:03.845 --> 00:03:05.079
When I first learned about these things,

00:03:05.079 --> 00:03:05.925
they always threw me off

00:03:05.925 --> 00:03:07.910
because here, this Leibniz notation,

00:03:07.910 --> 00:03:09.547
you have the great intution of, you know,

00:03:09.547 --> 00:03:11.013
nudging the x and nudging the f.

00:03:11.013 --> 00:03:13.267
But you kind of lose
that when you do this.

00:03:13.267 --> 00:03:15.608
But it makes sense if you think of this

00:03:15.608 --> 00:03:17.635
partial, partial x as being an operator

00:03:17.635 --> 00:03:19.409
and you're just applying it twice.

00:03:19.409 --> 00:03:21.073
And over here, the way
that that would look,

00:03:21.073 --> 00:03:22.331
it's a little bit funny.

00:03:22.331 --> 00:03:25.014
Because you still have that
partial squared f on top.

00:03:25.014 --> 00:03:26.357
But then on the bottom,

00:03:26.357 --> 00:03:30.987
you write partial y, partial x.

00:03:30.987 --> 00:03:32.806
And, you know, I'm putting
them in these order

00:03:32.806 --> 00:03:35.061
just because it's as if I
wrote it that way, right.

00:03:35.061 --> 00:03:37.247
This reflects the fact that
first I did the x derivative.

00:03:37.247 --> 00:03:39.414
Then I did the y derivative.

00:03:39.414 --> 00:03:41.506
And you could do this on this side also.

00:03:41.506 --> 00:03:42.842
And this might feel tedious,

00:03:42.842 --> 00:03:44.607
but it's actually kind of worth doing

00:03:44.607 --> 00:03:46.695
for a result that we end up seeing here

00:03:46.695 --> 00:03:49.795
that I find a little bit
surprising, actually.

00:03:49.795 --> 00:03:51.885
So here, if we go down the path

00:03:51.885 --> 00:03:54.717
of doing, in this case,
like a partial derivative

00:03:54.717 --> 00:03:56.145
with respect to x.

00:03:56.145 --> 00:03:57.244
And, you know, you're thinking of this

00:03:57.244 --> 00:03:59.715
as being applied to your
original partial derivative

00:03:59.715 --> 00:04:00.941
with respect to y.

00:04:02.061 --> 00:04:04.923
It looks here, it says sine
of x looks like a variable.

00:04:04.923 --> 00:04:06.545
Two y looks like a constant.

00:04:06.545 --> 00:04:07.923
So what we end up getting

00:04:07.923 --> 00:04:11.023
is derivative of sine of x, cosine x.

00:04:13.063 --> 00:04:14.730
Multiplied by that two y.

00:04:18.500 --> 00:04:21.251
And a pretty cool thing
worth pointing out here

00:04:21.251 --> 00:04:23.380
that maybe you take it for granted.

00:04:23.380 --> 00:04:24.832
Maybe you think it's as surprising

00:04:24.832 --> 00:04:26.148
as I did when I first saw it.

00:04:26.148 --> 00:04:28.493
Both of these turn out to be equal, right.

00:04:28.493 --> 00:04:30.002
Even though it was a very different way

00:04:30.002 --> 00:04:30.813
that we got there, right?

00:04:30.813 --> 00:04:32.757
You first take the partial
derivative with respect to x

00:04:32.757 --> 00:04:34.423
and you get cosine x, y squared.

00:04:34.423 --> 00:04:36.920
Which looks very different
from sine x, two y.

00:04:36.920 --> 00:04:38.503
And then when you take the derivative

00:04:38.503 --> 00:04:40.011
with respect to y,

00:04:41.201 --> 00:04:42.758
you know, you get a certain value.

00:04:42.758 --> 00:04:44.582
And when you go down the other path,

00:04:44.582 --> 00:04:46.526
you also get that same value.

00:04:46.526 --> 00:04:48.151
And maybe the way that you write this

00:04:48.151 --> 00:04:50.139
is that you'd say

00:04:50.139 --> 00:04:52.952
Let me just copy this guy over here.

00:04:54.342 --> 00:04:57.022
And what you might say is that

00:04:59.622 --> 00:05:01.214
the partial derivative of f.

00:05:01.214 --> 00:05:02.592
When you do it the other way around,

00:05:02.592 --> 00:05:04.367
when instead of doing x and then y,

00:05:04.367 --> 00:05:08.329
you do y and then x.

00:05:08.329 --> 00:05:09.618
Partial x.

00:05:09.618 --> 00:05:11.390
That these guys are equal to each other.

00:05:11.390 --> 00:05:13.252
And that's a pretty cool result.

00:05:14.202 --> 00:05:15.595
And maybe in this case,

00:05:15.595 --> 00:05:17.372
given that the original function

00:05:17.372 --> 00:05:18.895
just looks like the product of two things,

00:05:18.895 --> 00:05:20.776
you can kind of resaon
through why it's the case.

00:05:20.776 --> 00:05:24.366
But what's surprising is that
this turns out to be true

00:05:24.366 --> 00:05:26.092
for, I mean, not all functions.

00:05:26.092 --> 00:05:28.155
There's actually a certain criterion.

00:05:28.155 --> 00:05:29.589
There's a special theorem,

00:05:29.589 --> 00:05:31.864
it's called Schwarz's theorem.

00:05:32.824 --> 00:05:36.145
Where if the second partial derivatives

00:05:36.145 --> 00:05:38.405
of your function are continuous
at the relevant point,

00:05:38.405 --> 00:05:40.289
that's the circumstance
for this being true.

00:05:40.289 --> 00:05:42.095
But for all intents and purposes,

00:05:42.095 --> 00:05:43.975
the kind of functions you
can expect to run into,

00:05:43.975 --> 00:05:45.464
this is the case.

00:05:45.464 --> 00:05:48.357
This order of partial
derivatives doesn't matter.

00:05:48.357 --> 00:05:50.344
Truth turns out to hold.

00:05:50.344 --> 00:05:51.866
Which is actually pretty cool.

00:05:51.866 --> 00:05:53.424
And I'd encourage you to play around

00:05:53.424 --> 00:05:54.353
with some other functions.

00:05:54.353 --> 00:05:56.168
Just come up with any
multivariable function,

00:05:56.168 --> 00:05:57.383
maybe a little bit more complicated

00:05:57.383 --> 00:06:00.127
than just multiplying two
separate things there,

00:06:00.127 --> 00:06:01.669
and see that it's true.

00:06:01.669 --> 00:06:02.913
And maybe try to convince yourself

00:06:02.913 --> 00:06:04.394
why it's true in certain cases.

00:06:04.394 --> 00:06:07.059
I think that ought to actually
be a really good exercise.

00:06:07.059 --> 00:06:08.481
And just before I go,

00:06:08.481 --> 00:06:10.169
one thing that I should probably mention,

00:06:10.169 --> 00:06:14.116
a bit of notation that
people will commonly use.

00:06:14.116 --> 00:06:15.757
With the second partial derivative,

00:06:15.757 --> 00:06:17.485
sometimes instead of saying

00:06:17.485 --> 00:06:19.793
partial squared f, partial x squared,

00:06:19.793 --> 00:06:22.894
they'll just write it as
partial and then x, x.

00:06:22.894 --> 00:06:25.696
And over here, this would be partial.

00:06:25.696 --> 00:06:27.597
Let's see, first you
did it with x, then y.

00:06:27.597 --> 00:06:30.045
So over here you do it first x and then y.

00:06:30.045 --> 00:06:31.418
Kind of the order of these reverses.

00:06:31.418 --> 00:06:32.800
Because you're reading left to right.

00:06:32.800 --> 00:06:33.909
But when you do it with this,

00:06:33.909 --> 00:06:35.225
you're kind of reading right to left

00:06:35.225 --> 00:06:36.479
for how you multiply it in.

00:06:36.479 --> 00:06:38.593
Which would mean that this guy,

00:06:38.593 --> 00:06:40.349
let's see, this guy over here.

00:06:40.349 --> 00:06:41.721
Now he would be partial.

00:06:41.721 --> 00:06:42.960
First you did the y,

00:06:42.960 --> 00:06:44.211
and then you did the x.

00:06:44.211 --> 00:06:47.816
So those two guys are
just different notations

00:06:47.816 --> 00:06:48.896
for the same thing.

00:06:48.896 --> 00:06:50.666
I mean, that can make it a
little bit more convenient

00:06:50.666 --> 00:06:52.320
when you don't want to write out

00:06:52.320 --> 00:06:55.932
the entire partial squared f
divided by partial x squared

00:06:55.932 --> 00:06:57.684
or things like that.

00:06:59.324 --> 00:07:01.324
And with that, I'll call it an end.


Partial derivatives, introduction

WEBVTT
Kind: captions
Language: en

00:00:00.222 --> 00:00:01.110
- [Voiceover] So, let's say I have

00:00:01.110 --> 00:00:04.129
some multi-variable function like F of XY.

00:00:04.129 --> 00:00:06.428
So, they'll have a two variable input,

00:00:06.428 --> 00:00:08.065
is equal to, I don't know,

00:00:08.065 --> 00:00:11.663
X squared times Y, plus sin(Y).

00:00:11.663 --> 00:00:13.125
So, allow for just a single number.

00:00:13.125 --> 00:00:14.994
It's a scalar valued function.

00:00:14.994 --> 00:00:16.770
Question is, how do we take the derivative

00:00:16.770 --> 00:00:18.733
of an expression like this?

00:00:18.733 --> 00:00:21.089
And there's a certain method
called a partial derivative,

00:00:21.089 --> 00:00:23.017
which is very similar
to ordinary derivatives

00:00:23.017 --> 00:00:24.677
and I kinda wanna show
how they're secretly

00:00:24.677 --> 00:00:25.815
the same thing.

00:00:25.815 --> 00:00:27.951
So, to do that, let me
just remind ourselves

00:00:27.951 --> 00:00:32.050
of how we interpret the notation
for ordinary derivatives.

00:00:32.050 --> 00:00:35.579
So, if you have something
like F(X)=X squared,

00:00:35.579 --> 00:00:38.075
and let's say you wanna
take its derivative,

00:00:38.075 --> 00:00:41.547
and I'll use the Leibniz notation here, df/dx,

00:00:41.547 --> 00:00:44.712
and let's evaluate it at two, let's say.

00:00:44.712 --> 00:00:46.593
I really like this notation
because it's suggestive

00:00:46.593 --> 00:00:47.591
of what's going on.

00:00:47.591 --> 00:00:49.674
If we sketch out a graph,

00:00:51.318 --> 00:00:54.975
so this axis represents our output,

00:00:54.975 --> 00:00:56.752
this over here represents our input,

00:00:56.752 --> 00:00:59.549
and X squared has a certain
parabolic shape to it.

00:00:59.549 --> 00:01:01.036
Something like that.

00:01:01.036 --> 00:01:05.552
Then we go to the input,
x equals one, two.

00:01:05.552 --> 00:01:07.932
This little DX here, I like to interpret

00:01:07.932 --> 00:01:10.589
as just a little nudge in the X direction.

00:01:10.589 --> 00:01:12.666
And it's kind of the size of that nudge.

00:01:12.666 --> 00:01:14.666
And then DF,

00:01:15.714 --> 00:01:18.212
is the resulting change in the output

00:01:18.212 --> 00:01:21.254
after you make that initial little nudge.

00:01:21.254 --> 00:01:22.658
So, it's this resulting change.

00:01:22.658 --> 00:01:23.954
And when you're thinking
n terms of graphs,

00:01:23.954 --> 00:01:24.787
this is slope.

00:01:24.787 --> 00:01:26.640
You kind of have this rise over run

00:01:26.640 --> 00:01:28.161
for your ratio between the tiny change

00:01:28.161 --> 00:01:31.122
of the output that's caused
by a tiny change in the input.

00:01:31.122 --> 00:01:33.466
And of course, this is
dependent on where you start.

00:01:33.466 --> 00:01:35.766
Over here we have X=2.

00:01:35.766 --> 00:01:37.804
But, you could also think
about this without graphs

00:01:37.804 --> 00:01:39.243
if you really wanted to.

00:01:39.243 --> 00:01:41.124
You might just think about,

00:01:41.124 --> 00:01:43.563
your input space as just a number line

00:01:43.563 --> 00:01:47.730
and your output space,
also as just a number line,

00:01:48.810 --> 00:01:50.285
the output of F over here.

00:01:50.285 --> 00:01:53.083
And really, you're just thinking
of somehow mapping numbers

00:01:53.083 --> 00:01:55.439
from here onto the second line.

00:01:55.439 --> 00:01:56.925
And in that case, your initial nudge,

00:01:56.925 --> 00:01:59.190
your initial little DX,

00:01:59.190 --> 00:02:01.883
would be some nudge on that number line.

00:02:01.883 --> 00:02:02.905
And you're wondering how

00:02:02.905 --> 00:02:05.142
that influences the function itself.

00:02:05.142 --> 00:02:08.357
So maybe that causes a nudge
that's four times as big

00:02:08.357 --> 00:02:13.188
and that would mean your
derivative is four at that point.

00:02:13.188 --> 00:02:15.000
So, the reason I'm talking about this is

00:02:15.000 --> 00:02:16.822
because over in the multi-variable world,

00:02:16.822 --> 00:02:19.905
we can pretty much do the same thing.

00:02:21.739 --> 00:02:26.423
You can write df/dx and
interpret that as saying,

00:02:26.423 --> 00:02:29.510
hey how does a tiny change in
the input in the X direction

00:02:29.510 --> 00:02:31.577
influence the output.

00:02:31.577 --> 00:02:33.457
But, this time, the way
that you might visualize it,

00:02:33.457 --> 00:02:35.141
you'd be thinking of your input space.

00:02:35.141 --> 00:02:38.380
Here, I'll draw it down
here, as the XY plane.

00:02:38.380 --> 00:02:40.157
So, this time, this is not
gonna be graphing the function,

00:02:40.157 --> 00:02:43.512
this is, every point on
the plane is an input.

00:02:43.512 --> 00:02:45.104
And let's say you were evaluating this

00:02:45.104 --> 00:02:47.187
at a point like one, two.

00:02:49.274 --> 00:02:53.118
In that case, you'd go over
to the input that's one

00:02:53.118 --> 00:02:56.079
and then two, and then you'd say, okay,

00:02:56.079 --> 00:02:57.797
so this tiny nudge in the input,

00:02:57.797 --> 00:03:01.964
this tiny change DX, how does
that influence the output.

00:03:03.103 --> 00:03:04.380
And in this case the output, I mean,

00:03:04.380 --> 00:03:05.526
it's still just a number.

00:03:05.526 --> 00:03:08.057
So maybe we go off to the side here

00:03:08.057 --> 00:03:12.446
and we draw just like a
number line as our output.

00:03:12.446 --> 00:03:14.548
And somehow we're thinking
about the function

00:03:14.548 --> 00:03:17.137
as mapping points on the
plane to the number line.

00:03:17.137 --> 00:03:19.076
So you'd say, okay, that's your DX,

00:03:19.076 --> 00:03:21.177
how much does it change the output?

00:03:21.177 --> 00:03:23.394
And, maybe this time it
changes it negatively.

00:03:23.394 --> 00:03:25.066
It depends on your function.

00:03:25.066 --> 00:03:26.715
And that would be your DF.

00:03:26.715 --> 00:03:28.874
And you can also do this
with the Y variable, right.

00:03:28.874 --> 00:03:32.541
There's no reason that
you can't say DF, DY,

00:03:34.268 --> 00:03:37.834
and evaluate at that same point, one, two.

00:03:37.834 --> 00:03:40.457
And interpret totally the same way.

00:03:40.457 --> 00:03:43.394
Except, this time your DY would be

00:03:43.394 --> 00:03:46.355
a change in the Y direction.

00:03:46.355 --> 00:03:48.421
So maybe I should really
emphasize here that

00:03:48.421 --> 00:03:52.822
that DX is a change in
the X direction here

00:03:52.822 --> 00:03:54.866
and that DY is a change
in the Y direction.

00:03:54.866 --> 00:03:57.721
And maybe, when you change
your F according to Y,

00:03:57.721 --> 00:03:58.801
it does something different.

00:03:58.801 --> 00:04:02.851
Maybe, the output increases
and it increases by a lot,

00:04:02.851 --> 00:04:04.128
it's more sensitive to Y.

00:04:04.128 --> 00:04:06.401
Again, it depends on the function.

00:04:06.401 --> 00:04:08.625
And I'll show you how you can
compute something like this

00:04:08.625 --> 00:04:10.065
in just a moment here.

00:04:10.065 --> 00:04:12.340
But, first there's kind of
an annoying thing associated

00:04:12.340 --> 00:04:14.605
with partial derivatives,
where we don't write them

00:04:14.605 --> 00:04:17.228
with D's in DX/DF.

00:04:17.228 --> 00:04:19.293
People came up with this new notation,

00:04:19.293 --> 00:04:21.835
mostly just to emphasize to the reader

00:04:21.835 --> 00:04:23.075
of your equation that it's

00:04:23.075 --> 00:04:25.119
a multi-variable function involved.

00:04:25.119 --> 00:04:28.579
And what you do, is
you say, you write a D,

00:04:28.579 --> 00:04:30.680
but it's got kind of a curl at the top.

00:04:30.680 --> 00:04:33.072
It's this new symbol and
people will often read it

00:04:33.072 --> 00:04:34.279
as partial.

00:04:34.279 --> 00:04:36.589
So, you might read like
partial F, partial Y.

00:04:36.589 --> 00:04:37.710
If you're wondering, by the way,

00:04:37.710 --> 00:04:39.213
why we call these partial derivatives,

00:04:39.213 --> 00:04:41.350
it's sort of like, this
doesn't tell the full story

00:04:41.350 --> 00:04:44.112
of how F changes 'cause it only
cares about the X direction.

00:04:44.112 --> 00:04:46.794
Neither does this, this only
cares about the Y direction.

00:04:46.794 --> 00:04:50.997
So, each one is only a
small part of the story.

00:04:50.997 --> 00:04:53.749
So, let's actually evaluate
something like this.

00:04:53.749 --> 00:04:56.071
I'm gonna go ahead and
clear the board over here.

00:04:56.071 --> 00:04:58.988
I think the one-dimensional analogy

00:04:59.937 --> 00:05:03.104
is something we probably have already.

00:05:04.933 --> 00:05:06.600
So, little remnants.

00:05:08.647 --> 00:05:10.690
So, if you're actually
evaluating something like this,

00:05:10.690 --> 00:05:12.240
here, I'll write it up here again up here.

00:05:12.240 --> 00:05:15.607
Partial derivative of
F, with respect to X,

00:05:15.607 --> 00:05:17.046
and we're doing it at one, two.

00:05:17.046 --> 00:05:19.926
It only cares about
movement in the X direction,

00:05:19.926 --> 00:05:21.540
so it's treating Y as a constant.

00:05:21.540 --> 00:05:23.501
It doesn't even care about
the fact that Y changes.

00:05:23.501 --> 00:05:26.264
As far as it's concerned,
Y is always equal to two.

00:05:26.264 --> 00:05:28.947
So, we can just plug
that in ahead of time.

00:05:28.947 --> 00:05:31.025
So, I'm gonna say partial, partial X,

00:05:31.025 --> 00:05:32.198
this is another way you might write it,

00:05:32.198 --> 00:05:33.800
put the expression in here.

00:05:33.800 --> 00:05:35.646
And I'll say X squared,
but instead of writing Y,

00:05:35.646 --> 00:05:40.018
I'm just gonna plug in that
constant ahead of time.

00:05:40.018 --> 00:05:42.050
'Cause when you're only
moving in the X direction,

00:05:42.050 --> 00:05:45.207
this is kind of how the
multi-variable function

00:05:45.207 --> 00:05:46.380
sees the world.

00:05:46.380 --> 00:05:47.529
And I'll just keep a little note

00:05:47.529 --> 00:05:51.685
that we're evaluating this
whole thing at X equals one.

00:05:51.685 --> 00:05:53.845
And here, this is actually
just an ordinary derivative.

00:05:53.845 --> 00:05:55.842
This is an expression that's an X,

00:05:55.842 --> 00:05:57.642
you're asking how it changes

00:05:57.642 --> 00:06:00.010
as you shift around X and
you know how to do this.

00:06:00.010 --> 00:06:03.830
This is just taking the derivative

00:06:03.830 --> 00:06:05.697
of X square times two

00:06:05.697 --> 00:06:09.794
is gonna be 4x 'cause
X squared goes to 2x.

00:06:09.794 --> 00:06:11.281
And then the derivative of a constant,

00:06:11.281 --> 00:06:15.146
sin of two is just a constant, is zero.

00:06:15.146 --> 00:06:17.520
And of course we're evaluating
this at X equals one,

00:06:17.520 --> 00:06:20.791
so your overall answer is gonna be four.

00:06:20.791 --> 00:06:22.033
And as for practice, let's also do

00:06:22.033 --> 00:06:24.111
that with derivative with respect to Y.

00:06:24.111 --> 00:06:27.385
So, we look over here, I'm
gonna write the same thing.

00:06:27.385 --> 00:06:29.254
You're taking the partial derivative of F

00:06:29.254 --> 00:06:31.391
with respect to Y.

00:06:31.391 --> 00:06:34.073
We're evaluating it at
the same point one, two.

00:06:34.073 --> 00:06:39.024
This time it doesn't care about
movement in the X direction.

00:06:39.024 --> 00:06:40.676
So, as far as it's concerned,

00:06:40.676 --> 00:06:43.021
that X just stays constant at one.

00:06:43.021 --> 00:06:47.188
So, we'd write one squared
times Y, plus sin(Y).

00:06:49.221 --> 00:06:51.270
Sin(Y).

00:06:51.270 --> 00:06:52.103
And you're saying, oh,

00:06:52.103 --> 00:06:55.492
I'm keeping track of this at Y=2.

00:06:55.492 --> 00:06:58.094
So, it's kind of, you're
evaluating at Y=2.

00:06:58.094 --> 00:07:01.275
When you take the
derivative, this is just 1xY.

00:07:01.275 --> 00:07:03.399
So the derivative is one.

00:07:03.399 --> 00:07:07.566
This over here, the
derivative is cosine, cos(Y).

00:07:10.377 --> 00:07:14.347
Again, we're evaluating
this whole thing at Y=2.

00:07:14.347 --> 00:07:15.567
So, you're overall answer,

00:07:15.567 --> 00:07:18.126
it would be 1+cos(2).

00:07:18.126 --> 00:07:20.575
I'm not sure what the value of cos(2) is

00:07:20.575 --> 00:07:21.597
off the top of my head,

00:07:21.597 --> 00:07:23.443
but that would be your answer.

00:07:23.443 --> 00:07:27.401
And, this is a partial
derivative at a point,

00:07:27.401 --> 00:07:29.946
but a lot of times, you're
not asked to just compute it

00:07:29.946 --> 00:07:31.894
at a point, what you
want is a general formula

00:07:31.894 --> 00:07:34.421
that tells you, hey, plug in any point XY

00:07:34.421 --> 00:07:36.477
and it should spit out the answer.

00:07:36.477 --> 00:07:39.217
So, let me just kinda go
over how you would do that.

00:07:39.217 --> 00:07:40.564
It's actually very similar,

00:07:40.564 --> 00:07:42.537
but this time, instead of
plugging in the constant

00:07:42.537 --> 00:07:44.360
ahead of time, we just have to pretend

00:07:44.360 --> 00:07:45.544
that it's a constant.

00:07:45.544 --> 00:07:49.711
So let me make a little bit
of space for ourselves here.

00:07:53.628 --> 00:07:56.189
We don't need any of this anymore.

00:07:56.189 --> 00:08:00.356
I'm gonna leave the partial
partial F, partial partial Y.

00:08:01.378 --> 00:08:05.545
We want this as a more
general function of X and Y.

00:08:09.644 --> 00:08:11.664
Well we kind of do the same thing.

00:08:11.664 --> 00:08:13.290
We're gonna say that this is derivative

00:08:13.290 --> 00:08:14.334
with respect to X,

00:08:14.334 --> 00:08:16.389
and I'm using partials
just to kind of emphasize

00:08:16.389 --> 00:08:18.293
that it's a partial derivative.

00:08:18.293 --> 00:08:20.011
And now, we'd write X squared

00:08:20.011 --> 00:08:24.923
and then kind of emphasize that
it's a constant value of Y,

00:08:24.923 --> 00:08:28.463
plus the sin, and again,

00:08:28.463 --> 00:08:29.508
I'll say Y.

00:08:29.508 --> 00:08:33.281
And here, I'm writing the variable Y,

00:08:33.281 --> 00:08:34.999
but we have to pretend
like it's a constant,

00:08:34.999 --> 00:08:36.644
you're pretending that you plugin two

00:08:36.644 --> 00:08:37.583
or something like that.

00:08:37.583 --> 00:08:39.324
And you still just take the derivative.

00:08:39.324 --> 00:08:40.485
So, in this case,

00:08:40.485 --> 00:08:42.587
the derivative of X
squared times a constant,

00:08:42.587 --> 00:08:45.170
is just 2x times that constant.

00:08:47.563 --> 00:08:50.407
And over here, the derivative
of a constant is always zero.

00:08:50.407 --> 00:08:52.427
So that's just always gonna be zero.

00:08:52.427 --> 00:08:54.029
So, this is your partial derivative

00:08:54.029 --> 00:08:55.736
as a more general formula.

00:08:55.736 --> 00:08:56.637
If you plugged in one,

00:08:56.637 --> 00:08:59.075
two to this, you'd get what we had before.

00:08:59.075 --> 00:09:00.468
And similarly, if you're doing this

00:09:00.468 --> 00:09:02.551
with partial F partial Y,

00:09:06.714 --> 00:09:08.131
we write down all of the same things,

00:09:08.131 --> 00:09:10.534
now you're taking it with respect to Y.

00:09:10.534 --> 00:09:14.701
And I'm just gonna copy
this formula here actually.

00:09:23.467 --> 00:09:25.035
But this time, we're considering all

00:09:25.035 --> 00:09:26.857
of the the X's to be constants.

00:09:26.857 --> 00:09:28.356
So, in this case, when
you take the derivative

00:09:28.356 --> 00:09:29.911
with respect to Y of
some kind of constant,

00:09:29.911 --> 00:09:31.909
constant squared is a constant,

00:09:31.909 --> 00:09:35.751
times Y, it's just gonna
equal that constant.

00:09:35.751 --> 00:09:37.632
So, this is gonna be X squared.

00:09:37.632 --> 00:09:40.174
And over here, you're taking
the derivative of sin(Y).

00:09:40.174 --> 00:09:44.341
There's no X's in there,
so that remains the sin(Y).

00:09:47.106 --> 00:09:48.387
Now this is a more general formula.

00:09:48.387 --> 00:09:50.430
If you plugged in one,
two, you would get one.

00:09:50.430 --> 00:09:53.776
I'm sorry that's cos(y).

00:09:53.776 --> 00:09:57.943
Cos(y) because we're taking a derivative.

00:09:59.869 --> 00:10:01.062
So, if you plugged in one, two,

00:10:01.062 --> 00:10:03.245
you would get 1+cos(1),

00:10:03.245 --> 00:10:04.603
which is what we had before.

00:10:04.603 --> 00:10:06.705
So this, this is really what you'll see

00:10:06.705 --> 00:10:08.829
for how to compute a partial derivative.

00:10:08.829 --> 00:10:10.965
You pretend that one of
the variables is constant

00:10:10.965 --> 00:10:12.544
and you take an ordinary derivative.

00:10:12.544 --> 00:10:13.437
And in the back of your mind,

00:10:13.437 --> 00:10:14.270
you're thinking this is

00:10:14.270 --> 00:10:16.484
because you're just
moving in one direction

00:10:16.484 --> 00:10:19.270
for the input and you're seeing
how that influences things.

00:10:19.270 --> 00:10:22.510
And then, you might move in
one direction for another input

00:10:22.510 --> 00:10:24.309
and see how that influences things.

00:10:24.309 --> 00:10:26.248
In the next video, I'll
show you what this means

00:10:26.248 --> 00:10:27.862
in terms of graphs and slopes,

00:10:27.862 --> 00:10:30.788
but it's important to understand
that graphs and slopes

00:10:30.788 --> 00:10:33.226
are not the only way to
understand derivatives

00:10:33.226 --> 00:10:34.609
because as soon as you start thinking

00:10:34.609 --> 00:10:36.513
about vector valued functions

00:10:36.513 --> 00:10:39.717
or functions with inputs of
higher dimensions than just two,

00:10:39.717 --> 00:10:41.992
you can no longer think
in terms of graphs.

00:10:41.992 --> 00:10:44.350
But, this idea of nudging
the input in some direction,

00:10:44.350 --> 00:10:46.579
seeing how that influences the output,

00:10:46.579 --> 00:10:49.690
and then taking the ratio
of that output nudge

00:10:49.690 --> 00:10:50.587
to the input nudge,

00:10:50.587 --> 00:10:52.305
that's a more general
way of viewing things.

00:10:52.305 --> 00:10:54.477
And that's gonna be very
helpful moving forward

00:10:54.477 --> 00:10:55.310
in multi-variable calc.


Directional derivatives and slope

WEBVTT
Kind: captions
Language: en

00:00:00.115 --> 00:00:02.783
- [Voiceover] Hello everyone,
what I wanna talk about here

00:00:02.783 --> 00:00:05.503
is how to interpret the
directional derivative

00:00:05.503 --> 00:00:07.079
in terms of graphs.

00:00:07.079 --> 00:00:09.031
I have here the graph of a function,

00:00:09.031 --> 00:00:11.051
a multivariable function,

00:00:11.051 --> 00:00:14.884
it's f of xy is equal
to x squared times y.

00:00:17.736 --> 00:00:18.804
In the last couple videos I talked

00:00:18.804 --> 00:00:21.104
about what the directional derivative is,

00:00:21.104 --> 00:00:22.404
how you can formally define it,

00:00:22.404 --> 00:00:24.404
how you can compute it using the gradient.

00:00:24.404 --> 00:00:28.328
Generally the setup
that you might have is,

00:00:28.328 --> 00:00:30.765
you have some kind of
vector, and this is a vector

00:00:30.765 --> 00:00:33.108
in the input space so in this case

00:00:33.108 --> 00:00:35.438
it's gonna be in the xy plane.

00:00:35.438 --> 00:00:39.021
In this case I'll just
take the vector 1 1.

00:00:41.840 --> 00:00:44.580
Okay? And the directional
derivative, which we

00:00:44.580 --> 00:00:46.764
denote by kind of taking
the gradient symbol,

00:00:46.764 --> 00:00:50.060
except you stick the name of
that vector down in the lower

00:00:50.060 --> 00:00:55.028
part there, the directional
derivative of your function,

00:00:55.028 --> 00:00:56.376
it'll still take the same input.

00:00:56.376 --> 00:00:58.884
This is kind of a measure
of how the function changes

00:00:58.884 --> 00:01:00.992
when the input moves in that direction.

00:01:00.992 --> 00:01:03.844
So I'll show you what I mean, I mean

00:01:03.844 --> 00:01:08.011
you could imagine slicing this
graph by some kind of plane

00:01:09.262 --> 00:01:11.701
but that plane doesn't
necessarily have to be parallel

00:01:11.701 --> 00:01:13.513
to the x or y axes.

00:01:13.513 --> 00:01:15.601
That's what we did for the
partial derivative, we took

00:01:15.601 --> 00:01:17.945
a plane that represented
the constant x value or

00:01:17.945 --> 00:01:19.990
the constant y value, but
this is gonna be a plane that

00:01:19.990 --> 00:01:22.777
kind of tells you what
movement in the direction of

00:01:22.777 --> 00:01:25.657
your vector looks like, and
like I have a number of other

00:01:25.657 --> 00:01:29.808
times I'm gonna go ahead and
slice the graph along that

00:01:29.808 --> 00:01:33.942
plane, and just to make it
clear, I'm gonna color in where

00:01:33.942 --> 00:01:37.034
the graph intersects that slice.

00:01:37.034 --> 00:01:39.914
This vector here, this little
v, you'll be thinking of it

00:01:39.914 --> 00:01:44.418
as living on the xy plane and
it's determining the direction

00:01:44.418 --> 00:01:47.066
of this plane that we're
slicing things with.

00:01:47.066 --> 00:01:48.946
On the xy plane you've got this vector,

00:01:48.946 --> 00:01:51.454
it's 1 1, it kind of points
to that diagonal direction,

00:01:51.454 --> 00:01:54.848
and then you take the whole
plane and you slice your graph.

00:01:54.848 --> 00:01:58.935
And if we want to interpret the
directional derivative here,

00:01:58.935 --> 00:02:01.523
I'm gonna go ahead and
fill it an actual value,

00:02:01.523 --> 00:02:04.856
so let's say we wanted to do it at -1 1,

00:02:06.947 --> 00:02:08.054
- 1, -1

00:02:08.054 --> 00:02:08.887
'cause I guess I chose a
plane that passes through the

00:02:08.887 --> 00:02:11.958
origin, so I've got to
make sure that the point

00:02:11.958 --> 00:02:14.046
I'm evaluating actually
goes along this plane,

00:02:14.046 --> 00:02:16.370
but you could imagine one that
points in the same direction,

00:02:16.370 --> 00:02:18.666
but you kind of slide it back and forth,

00:02:18.666 --> 00:02:22.918
if we're doing this, we can
interpret this as a slope,

00:02:22.918 --> 00:02:25.446
but you have to be very careful,
if you're gonna interpret

00:02:25.446 --> 00:02:28.166
this as a slope, it has to be
the case that you're dealing

00:02:28.166 --> 00:02:30.974
with a unit vector, that
the magnitude of your vector

00:02:30.974 --> 00:02:32.830
is equal to 1.

00:02:32.830 --> 00:02:34.526
I mean, it doesn't have to
be, you can kind of account

00:02:34.526 --> 00:02:36.802
for it later but it makes
it more easy to think about.

00:02:36.802 --> 00:02:39.126
If we're just thinking of a unit vector.

00:02:39.126 --> 00:02:42.002
When I go over here instead
of saying that it's 1 1,

00:02:42.002 --> 00:02:44.954
I'm gonna say it's whatever
vector points in the same

00:02:44.954 --> 00:02:48.574
direction but has a unit
length, and in this case that

00:02:48.574 --> 00:02:50.804
happens to be square root of 2 over 2,

00:02:50.804 --> 00:02:52.312
for each of the components.

00:02:52.312 --> 00:02:54.588
You can kind of think about
why that would be true

00:02:54.588 --> 00:02:58.491
by diagonal but this is a
vector with unit length,

00:02:58.491 --> 00:03:03.435
and its magnitude is 1, and
it points in that direction.

00:03:03.435 --> 00:03:06.859
If we're evaluating this
negative point like 1 1,

00:03:06.859 --> 00:03:09.192
we can draw that on the graph,
see where it actually is,

00:03:09.192 --> 00:03:12.444
and in this case it'll be,
oops, moving things about when

00:03:12.444 --> 00:03:13.852
I had a point.

00:03:13.852 --> 00:03:17.201
It'll be this point and if
you kinda look from above,

00:03:17.201 --> 00:03:19.034
you see that's -1, -1,

00:03:20.592 --> 00:03:22.964
and if we want the slope at that

00:03:22.964 --> 00:03:25.704
point, you're kinda thinking
of the tangent line here.

00:03:25.704 --> 00:03:27.840
Tangent line to that curve,
and we're wondering what its

00:03:27.840 --> 00:03:32.276
slope is, so, the reason that
the directional derivative

00:03:32.276 --> 00:03:36.876
is gonna give us this slope,
is because, another notation

00:03:36.876 --> 00:03:39.012
that might be kinda helpful
for what this directional

00:03:39.012 --> 00:03:42.949
derivative is, some people
will write partial f,

00:03:42.949 --> 00:03:44.116
and partial v.

00:03:48.012 --> 00:03:49.624
You can think about
that as taking a slight

00:03:49.624 --> 00:03:53.504
nudge in the direction of
v, so this would be a little

00:03:53.504 --> 00:03:57.220
nudge, a little partial
nudge in the direction of v.

00:03:57.220 --> 00:04:00.216
And then you're saying
"what changed in the value

00:04:00.216 --> 00:04:02.604
of the function that's then resulting?"

00:04:02.604 --> 00:04:04.556
"The height of the graph, does
it devalue the function?".

00:04:04.556 --> 00:04:07.248
As this initial change
approaches zero and the resulting

00:04:07.248 --> 00:04:10.688
change approaches zero as
well, that ratio, the ratio

00:04:10.688 --> 00:04:14.144
of the partial f to partial
v, is going to give the slope

00:04:14.144 --> 00:04:15.864
of this tangent line.

00:04:15.864 --> 00:04:19.208
Conceptually, that's kind
of a nicer notation, but the

00:04:19.208 --> 00:04:22.484
reason we use this other
notation is nabla sub v 1,

00:04:22.484 --> 00:04:25.152
is it's very indicative
of how you compute things

00:04:25.152 --> 00:04:27.439
once you need it computed.

00:04:27.439 --> 00:04:30.559
You take the gradient of f,
just the vector value function

00:04:30.559 --> 00:04:34.726
gradient of f, and take the
dot product with the vector.

00:04:37.200 --> 00:04:39.429
Let's actually do that,
just to see what this would

00:04:39.429 --> 00:04:41.797
look like, and I'll go ahead
and write it over here,

00:04:41.797 --> 00:04:43.909
use a different color.

00:04:43.909 --> 00:04:48.481
The gradient of f, first of
all, is a vector full of partial

00:04:48.481 --> 00:04:52.757
derivatives, it'll be the
partial derivative of f with

00:04:52.757 --> 00:04:56.425
respect to x and the partial
derivative of f with respect

00:04:56.425 --> 00:04:57.258
to y.

00:05:00.301 --> 00:05:03.065
When we actually evaluate
this, we take a look, partial

00:05:03.065 --> 00:05:06.771
derivative of f with respect
to x, x looks like the variable

00:05:06.771 --> 00:05:10.893
y I just a constant, so its
partial derivative is 2 times x

00:05:10.893 --> 00:05:12.413
times y.

00:05:12.413 --> 00:05:14.471
2 times x times y.

00:05:14.471 --> 00:05:15.730
but when we take the partial with

00:05:15.730 --> 00:05:18.597
respect to y, y now looks
like a variable, and x looks

00:05:18.597 --> 00:05:21.637
like a constant, derivative of
a constant times a variable,

00:05:21.637 --> 00:05:24.554
is just that constant x squared.

00:05:26.261 --> 00:05:30.372
And if we were to evaluate
this at the point -1, -1,

00:05:30.372 --> 00:05:34.539
then you can plug that in, 2
times -1 times -1 would be 2,

00:05:35.423 --> 00:05:38.756
and then negative 1 squared, would be 1.

00:05:39.937 --> 00:05:42.421
So that would be our gradient
at that point, which means

00:05:42.421 --> 00:05:46.833
if we want to evaluate gradient
of f times v, we could go

00:05:46.833 --> 00:05:49.829
over here, and say that's 2 1,

00:05:49.829 --> 00:05:54.437
'cause we evaluate the gradient
at the point we care about.

00:05:54.437 --> 00:05:58.091
And then the dot product,
with v itself in this case,

00:05:58.091 --> 00:06:00.841
root 2 over 2, and root 2 over 2.

00:06:03.949 --> 00:06:07.104
The answer that we get, we
multiply the fist two components

00:06:07.104 --> 00:06:11.124
together, 2 times root 2
over 2, then square it to 2,

00:06:11.124 --> 00:06:14.280
and then here we multiply the
second components together,

00:06:14.280 --> 00:06:19.047
and that's gonna be 1 times
root 2 over 2, root 2 over 2,

00:06:19.047 --> 00:06:21.248
and that would be our answer,
that would be our slope.

00:06:21.248 --> 00:06:25.404
But this only works if your
vector is a unit vector,

00:06:25.404 --> 00:06:27.656
and I showed this in the last
video where we talked about

00:06:27.656 --> 00:06:30.860
the formal definition of
the directional derivative.

00:06:30.860 --> 00:06:34.088
If you scale v by 2, and I
can do it here if instead of v

00:06:34.088 --> 00:06:37.084
you're talking about 2 v, so
I'll go ahead and make myself

00:06:37.084 --> 00:06:38.334
some room here.

00:06:44.087 --> 00:06:48.556
If you're taking the
directional derivative along 2 v

00:06:48.556 --> 00:06:51.690
of f, the way that we're
computing that, we're still taking

00:06:51.690 --> 00:06:55.857
the gradient of f, dot product
with 2 times your vector,

00:06:57.633 --> 00:07:00.069
and dot product, you
can pull out that too.

00:07:00.069 --> 00:07:04.597
This is just gonna double the
value of the entire thing.

00:07:04.597 --> 00:07:08.825
V, this started with v, it's
gonna be twice the value,

00:07:08.825 --> 00:07:11.193
the derivative will become
twice the value, but you don't

00:07:11.193 --> 00:07:13.397
necessarily want that because
you'd see this plane you

00:07:13.397 --> 00:07:16.809
sliced with, if instead of
doing it in the direction of v,

00:07:16.809 --> 00:07:20.085
the unit vector, you did in
the direction of 2 times v,

00:07:20.085 --> 00:07:22.893
it's the same plane, it's
the same slice you're taking,

00:07:22.893 --> 00:07:24.893
and you'd want that same
slope, so that's gonna mess

00:07:24.893 --> 00:07:25.749
everything up.

00:07:25.749 --> 00:07:27.793
This is super important if
you're thinking about things in

00:07:27.793 --> 00:07:30.311
the context of slope, one
thing that you could say is

00:07:30.311 --> 00:07:34.433
your formula for the slope of
a graph in the direction of v,

00:07:34.433 --> 00:07:37.593
is you take your directional
derivative, that dot product

00:07:37.593 --> 00:07:40.751
between f and v, and you just
always make sure to divide it

00:07:40.751 --> 00:07:44.955
by the magnitude of v,
divide it by that magnitude.

00:07:44.955 --> 00:07:47.555
That will always take care
of what you want, that's

00:07:47.555 --> 00:07:49.599
basically a way of making sure
that really, you're taking

00:07:49.599 --> 00:07:51.827
the directional derivative
in the direction of a certain

00:07:51.827 --> 00:07:52.987
unit vector.

00:07:52.987 --> 00:07:56.307
Some people even go so far
as to define the directional

00:07:56.307 --> 00:08:00.487
derivative to be this, to be
something where you normalize

00:08:00.487 --> 00:08:02.255
out the length of that vector.

00:08:02.255 --> 00:08:06.179
I don't really like that, but
I think that's because they're

00:08:06.179 --> 00:08:10.147
thinking of the slope context,
they're thinking of rates of

00:08:10.147 --> 00:08:12.747
change as being the slope of a graph.

00:08:12.747 --> 00:08:15.559
One thing I'd like to
emphasize as always, graphical

00:08:15.559 --> 00:08:18.903
intuition is good, and visual
intuition is always great,

00:08:18.903 --> 00:08:21.039
you should always be trying
to find a way to think about

00:08:21.039 --> 00:08:22.919
things visually, but with
multivariable functions,

00:08:22.919 --> 00:08:24.659
the graph isn't the only way.

00:08:24.659 --> 00:08:28.537
You can kind of more generally
think about just a nudge

00:08:28.537 --> 00:08:31.857
in the v direction, and in
the context where v doesn't

00:08:31.857 --> 00:08:35.085
have a length 1, the nudge
doesn't represent an actual size

00:08:35.085 --> 00:08:39.284
but it's a certain scaling
constant times that vector,

00:08:39.284 --> 00:08:41.164
you can look at the video on
the formal definition for the

00:08:41.164 --> 00:08:43.788
directional derivative, if
you want more details on that.

00:08:43.788 --> 00:08:46.460
But I do think this is actually
a good way to get a feel for

00:08:46.460 --> 00:08:48.923
what the directional
derivative is all about.


Transformations, part 1

WEBVTT
Kind: captions
Language: en

00:00:00.213 --> 00:00:01.413
- [Voiceover] So I have talked a lot

00:00:01.413 --> 00:00:03.038
about different ways
that you can visualize

00:00:03.038 --> 00:00:04.272
multivariable functions.

00:00:04.272 --> 00:00:07.083
Functions that will have
some kind of multidimensional

00:00:07.083 --> 00:00:08.917
input or output.

00:00:08.917 --> 00:00:10.520
These include three-dimensional graphs,

00:00:10.520 --> 00:00:11.336
which are very common.

00:00:11.336 --> 00:00:14.011
Contour maps, vector fields,
parametric functions.

00:00:14.011 --> 00:00:15.489
But here, I want to talk about

00:00:15.489 --> 00:00:16.854
one of my all-time favorite ways

00:00:16.854 --> 00:00:19.882
to think about functions,
which is as a transformation.

00:00:19.882 --> 00:00:22.366
So any time you have
some sort of function,

00:00:22.366 --> 00:00:24.179
if you're thinking very abstractly,

00:00:24.179 --> 00:00:26.394
I like to think that there's
some sort of input space.

00:00:26.394 --> 00:00:28.866
And I'll draw it as a blob, even though,

00:00:28.866 --> 00:00:30.235
you know, that could be
the real number line.

00:00:30.235 --> 00:00:31.028
So it should be a line,

00:00:31.028 --> 00:00:32.373
or it could be three-dimensional space.

00:00:32.373 --> 00:00:34.428
And then there's some
kind of output space.

00:00:34.428 --> 00:00:36.856
And again, I just very vaguely
think about it as this blob.

00:00:37.796 --> 00:00:40.053
But that could be, again,
the real number line,

00:00:40.053 --> 00:00:42.069
the x-y plane, a three-dimensional space.

00:00:42.069 --> 00:00:43.529
And the function is just some way

00:00:43.529 --> 00:00:45.542
of taking inputs to outputs.

00:00:45.542 --> 00:00:48.547
And every time that we're
trying to visualize something,

00:00:48.547 --> 00:00:50.403
like with a graph or a contour map,

00:00:50.403 --> 00:00:53.367
you're just trying to
associate input-ouput pairs.

00:00:53.367 --> 00:00:55.573
You know, if f inputs, you know,

00:00:55.573 --> 00:00:58.889
three gets mapped to the vector one-two.

00:00:58.889 --> 00:01:00.782
It's a question of how do you associate

00:01:00.782 --> 00:01:03.174
the number three with that vector one-two.

00:01:03.174 --> 00:01:04.800
And the thought behind transformations

00:01:04.800 --> 00:01:08.095
is that we're just gonna
watch the actual points

00:01:08.095 --> 00:01:10.943
of the input space move
to the output space.

00:01:10.943 --> 00:01:12.284
And I'll start with a simple example

00:01:12.284 --> 00:01:14.668
that's just a one-dimensional function.

00:01:14.668 --> 00:01:16.416
It will have a single variable input.

00:01:16.416 --> 00:01:18.068
And it will have a single variable output.

00:01:21.424 --> 00:01:23.813
So let's consider the function f of x

00:01:25.273 --> 00:01:27.743
is equal to x squared minus three.

00:01:30.024 --> 00:01:31.613
And, of course, the way
we're used to visualizing

00:01:31.613 --> 00:01:33.156
something like this,
it will be as a graph.

00:01:33.156 --> 00:01:34.863
You might kind of be thinking of something

00:01:34.863 --> 00:01:36.855
roughly parabolic that's
squished down by three.

00:01:36.855 --> 00:01:39.254
But here, I don't want to
think in terms of graphs.

00:01:39.254 --> 00:01:40.447
I just want to say,

00:01:40.447 --> 00:01:42.745
how do the inputs move to those outputs?

00:01:42.745 --> 00:01:45.621
So as an example, if you go to zero,

00:01:45.621 --> 00:01:48.159
when you plug in zero, you're
gonna get negative three.

00:01:48.159 --> 00:01:50.237
You know, zero squared minus three

00:01:50.237 --> 00:01:51.387
is equal to negative three.

00:01:52.977 --> 00:01:57.449
So somehow we want to watch
zero move to negative three.

00:01:57.449 --> 00:01:59.269
And then similarly, let's
say you plug in one,

00:01:59.269 --> 00:02:01.886
and you get one squared
minus three is negative two.

00:02:01.886 --> 00:02:05.744
So somehow we want to watch
one move to negative two.

00:02:06.854 --> 00:02:09.153
Just another example here.

00:02:09.153 --> 00:02:11.318
Let's say you are
plugging in three itself.

00:02:11.318 --> 00:02:16.172
So three squared minus three

00:02:16.172 --> 00:02:18.246
is nine minus three is six.

00:02:18.246 --> 00:02:19.771
So somehow in this transformation,

00:02:19.771 --> 00:02:23.319
we want to watch three
move to the number six.

00:02:26.917 --> 00:02:29.456
And with a little animation,
we can watch this happen.

00:02:29.456 --> 00:02:31.020
We can actually watch what it looks like

00:02:31.020 --> 00:02:32.269
for all these numbers to move

00:02:32.269 --> 00:02:34.451
to their corresponding outputs.

00:02:34.451 --> 00:02:35.850
So here we go.

00:02:35.850 --> 00:02:37.655
Each number will move over

00:02:38.705 --> 00:02:40.110
and land on its output.

00:02:41.360 --> 00:02:42.792
And I'll clear up the board here.

00:02:45.882 --> 00:02:48.415
So I kept track of what the
original input numbers are

00:02:48.415 --> 00:02:50.549
by just kind of writing them on top here.

00:02:50.549 --> 00:02:52.251
And that was a way of just
watching how it moves.

00:02:52.251 --> 00:02:52.946
And I'll play it again.

00:02:52.946 --> 00:02:54.701
Here, let's just watch where

00:02:56.031 --> 00:02:58.507
each number from the input space

00:02:58.507 --> 00:03:00.134
moves over to the output.

00:03:01.244 --> 00:03:03.229
And with single variable functions,

00:03:03.229 --> 00:03:05.127
this is a little bit nice
because it gives the sense

00:03:05.127 --> 00:03:06.842
of inputs moving to outputs.

00:03:06.842 --> 00:03:08.509
But where it gets fun is

00:03:08.509 --> 00:03:10.964
in the context of multivariable functions.

00:03:10.964 --> 00:03:14.204
So now, let me consider
a function that has

00:03:14.204 --> 00:03:18.108
a one-dimensional input and
a two-dimensional output.

00:03:18.108 --> 00:03:20.891
And specifically, it will be f of x

00:03:21.781 --> 00:03:25.066
is equal to cosine of x,

00:03:25.986 --> 00:03:28.185
cosine of x.

00:03:28.185 --> 00:03:29.548
And then the y component will be

00:03:29.548 --> 00:03:30.887
x sine of y.

00:03:33.816 --> 00:03:35.208
Sorry, x sine of x.

00:03:39.097 --> 00:03:41.405
So just to think about a couple examples.

00:03:41.405 --> 00:03:43.211
If you plug in something like zero,

00:03:43.211 --> 00:03:44.995
and think about where zero ought to go,

00:03:47.665 --> 00:03:49.297
you'd have f of zero

00:03:51.557 --> 00:03:53.925
is equal to cosine of zero is one.

00:03:54.875 --> 00:03:57.702
And then, zero times anything is zero.

00:03:57.702 --> 00:03:59.426
So somehow we're gonna watch zero

00:03:59.426 --> 00:04:01.876
move over to the point one-zero.

00:04:01.876 --> 00:04:04.106
And so this is where
we expect zero to land.

00:04:04.106 --> 00:04:06.315
And let's think about, like, pi.

00:04:06.315 --> 00:04:08.464
So f of pi.

00:04:14.007 --> 00:04:16.067
Then cosine of pi is negative one.

00:04:17.818 --> 00:04:19.697
This is gonna be pi multiplied by,

00:04:19.697 --> 00:04:21.240
and sine of pi is zero.

00:04:21.240 --> 00:04:22.419
So that will again be zero.

00:04:23.889 --> 00:04:26.087
So, you know, this little
guy is where zero lands.

00:04:26.087 --> 00:04:28.169
And we expect that this is gonna be where

00:04:28.169 --> 00:04:29.408
the value pi lands.

00:04:31.288 --> 00:04:32.657
And if we watch this take place,

00:04:32.657 --> 00:04:35.207
and we actually watch each
element of the input space

00:04:35.207 --> 00:04:37.602
move over to the output space,

00:04:37.602 --> 00:04:38.960
we get something like this.

00:04:44.470 --> 00:04:46.441
And again, this is just kind of a nice way

00:04:46.441 --> 00:04:47.854
to think about what's actually going on.

00:04:47.854 --> 00:04:49.089
You might ask questions about whether

00:04:49.089 --> 00:04:52.936
the space ends up getting
stretched or squished.

00:04:52.936 --> 00:04:55.548
And notice that this is
also what a parametric plot

00:04:55.548 --> 00:04:56.696
of this function would look like.

00:04:56.696 --> 00:04:58.275
If you interpret it as
a parametric function,

00:04:58.275 --> 00:04:59.799
this is what you get in the end.

00:04:59.799 --> 00:05:02.257
But whereas in parametrics plots,

00:05:02.257 --> 00:05:03.690
you lose input information,

00:05:03.690 --> 00:05:05.721
here you can kind of see where things move

00:05:05.721 --> 00:05:08.045
as you go from one to the other.

00:05:08.045 --> 00:05:09.118
And in the next video,

00:05:09.118 --> 00:05:11.190
I'm gonna talk about how
you can interpret functions

00:05:11.190 --> 00:05:12.668
with a two-dimensional input

00:05:12.668 --> 00:05:13.986
and a two-dimensional output

00:05:13.986 --> 00:05:15.591
as a transformation.