Lagrange multipliers, using tangency to solve constrained optimization

WEBVTT
Kind: captions
Language: en

00:00:00.339 --> 00:00:01.694
- [Instructor] In the
last video I introduced

00:00:01.694 --> 00:00:03.860
a constrained optimization problem

00:00:03.860 --> 00:00:05.841
where we were trying to
maximize this function,

00:00:05.841 --> 00:00:08.318
f of x, y equals x squared times y,

00:00:08.318 --> 00:00:11.225
but subject to a constraint
that your values of x and y

00:00:11.225 --> 00:00:15.251
have to satisfy x squared
plus y squared equals one.

00:00:15.251 --> 00:00:17.053
And the way we were visualizing this

00:00:17.053 --> 00:00:20.108
was to look at the x, y
plane where this circle here

00:00:20.108 --> 00:00:21.530
represents our constraint.

00:00:21.530 --> 00:00:23.195
All of the points that make up this set,

00:00:23.195 --> 00:00:24.998
x squared plus y squared equals one,

00:00:24.998 --> 00:00:29.689
and then this curvy line here
is one of the contours of f,

00:00:29.689 --> 00:00:34.334
meaning, we're setting f of
x, y equal to some constant.

00:00:34.334 --> 00:00:37.588
And then I was varying
around that constant c.

00:00:37.588 --> 00:00:40.059
So for high values of c,
the contour would look

00:00:40.059 --> 00:00:41.743
something like this,
this is where the value

00:00:41.743 --> 00:00:44.011
of x squared times y is big.

00:00:44.011 --> 00:00:46.412
And then for small values of c,

00:00:46.412 --> 00:00:48.002
the contours would look like this.

00:00:48.002 --> 00:00:50.460
So all the points on this
line would be f of x, y

00:00:50.460 --> 00:00:54.434
equals like 0.01 in this
case, something like that.

00:00:54.434 --> 00:00:57.185
Then the way to think about
maximizing this function

00:00:57.185 --> 00:00:59.292
is to try to increase that value of c

00:00:59.292 --> 00:01:02.965
as much as you can without
it falling off the circle.

00:01:02.965 --> 00:01:04.663
And the key observation is

00:01:04.663 --> 00:01:06.391
that happens when they're tangent.

00:01:06.391 --> 00:01:08.079
So, you know, you might
kind of draw this out

00:01:08.079 --> 00:01:09.797
in a little sketch and
say there's some curve

00:01:09.797 --> 00:01:11.539
representing your constraint,
which in this case

00:01:11.539 --> 00:01:13.727
would be, you know, where our circle is.

00:01:13.727 --> 00:01:15.752
And then the curve
representing the contour

00:01:15.752 --> 00:01:17.856
would just kiss that curve,

00:01:17.856 --> 00:01:20.888
just barely touch it in some way.

00:01:20.888 --> 00:01:23.540
Now, that's pretty, but in
terms of solving the problem,

00:01:23.540 --> 00:01:26.040
we still have some work to do.

00:01:26.898 --> 00:01:29.543
And the main tool we're gonna
use here is the gradient.

00:01:29.543 --> 00:01:31.702
So let me go ahead and draw
a lot more contour lines

00:01:31.702 --> 00:01:34.222
than there already are
for x squared times y.

00:01:34.222 --> 00:01:35.649
So this is many of the contour lines,

00:01:35.649 --> 00:01:39.816
and I'll draw the gradient
field, the gradient field of f.

00:01:42.101 --> 00:01:43.507
So I've made a video
about the relationship

00:01:43.507 --> 00:01:45.774
between the gradient and contour lines.

00:01:45.774 --> 00:01:48.566
And the upshot of it is
that these gradient vectors,

00:01:48.566 --> 00:01:50.645
every time they pass
through a contour line,

00:01:50.645 --> 00:01:52.799
they're perpendicular to it.

00:01:52.799 --> 00:01:53.860
And the basic reason for that

00:01:53.860 --> 00:01:55.826
is if you walk along the contour line,

00:01:55.826 --> 00:01:57.488
the function isn't changing value,

00:01:57.488 --> 00:01:59.392
so if you want it to change most rapidly,

00:01:59.392 --> 00:02:01.044
you know, it kind of makes
sense you should walk

00:02:01.044 --> 00:02:02.936
in the perpendicular direction,

00:02:02.936 --> 00:02:05.270
so that no component of
the walk that you're taking

00:02:05.270 --> 00:02:07.770
is, you know, useless, is along the line

00:02:07.770 --> 00:02:09.972
where the function doesn't change.

00:02:09.972 --> 00:02:11.431
But again, there's a whole video on that

00:02:11.431 --> 00:02:15.579
that's worthy checking out
if this feels unfamiliar.

00:02:15.579 --> 00:02:17.268
For our purposes, what it means is that

00:02:17.268 --> 00:02:19.691
when we're considering
this point of tangency,

00:02:19.691 --> 00:02:24.434
the gradient of f at that
point is gonna be some vector

00:02:24.434 --> 00:02:27.398
perpendicular to both
the curves at that point.

00:02:27.398 --> 00:02:30.298
So that little vector
represents the gradient of f

00:02:30.298 --> 00:02:33.329
at this point on the plane.

00:02:33.329 --> 00:02:34.512
And we can do something very similar

00:02:34.512 --> 00:02:36.361
to understand the other curve.

00:02:36.361 --> 00:02:37.939
Right now I've just
written it as a constraint,

00:02:37.939 --> 00:02:39.758
x squared plus y squared equals one.

00:02:39.758 --> 00:02:41.625
But you know, to give
that function a name,

00:02:41.625 --> 00:02:44.875
let's say we've defined g of x, y to be

00:02:46.921 --> 00:02:48.859
x squared plus y squared,

00:02:48.859 --> 00:02:50.463
x squared plus y squared.

00:02:50.463 --> 00:02:52.979
In that case, this
constraint is pretty much

00:02:52.979 --> 00:02:56.198
just one of the contour
lines for the function g,

00:02:56.198 --> 00:02:57.289
and we can take a look at that.

00:02:57.289 --> 00:02:58.867
If we go over here and we look at

00:02:58.867 --> 00:03:02.165
all of the other contour
lines for this function g,

00:03:02.165 --> 00:03:03.614
and it should make sense
that they're circles,

00:03:03.614 --> 00:03:08.009
because this function is
x squared plus y squared.

00:03:08.009 --> 00:03:10.263
And if we took a look
at the gradient of g,

00:03:10.263 --> 00:03:14.180
and we go over and ask
about the gradient of g,

00:03:15.368 --> 00:03:16.847
it has that same property,

00:03:16.847 --> 00:03:18.097
that every gradient vector,

00:03:18.097 --> 00:03:19.797
if it passes through a contour line,

00:03:19.797 --> 00:03:21.688
is perpendicular to it.

00:03:21.688 --> 00:03:23.197
So over on our drawing here,

00:03:23.197 --> 00:03:26.257
the gradient vector of g
would also be perpendicular

00:03:26.257 --> 00:03:27.663
to both these curves.

00:03:27.663 --> 00:03:28.648
And you know, maybe in this case,

00:03:28.648 --> 00:03:31.574
it's not as long as the gradient
of f, or maybe it's longer.

00:03:31.574 --> 00:03:33.563
There's no reason that it
would be the same length,

00:03:33.563 --> 00:03:37.574
but the important fact is
that it's proportional.

00:03:37.574 --> 00:03:39.338
And the way that we're
gonna write this in formulas

00:03:39.338 --> 00:03:42.695
is to say that the
gradient of f evaluated,

00:03:42.695 --> 00:03:45.741
let's see, evaluated at
whatever the maximizing value

00:03:45.741 --> 00:03:48.231
of x and y are, so we should
give that a name probably.

00:03:48.231 --> 00:03:52.139
Maybe x sub m, y sub
m, the specific values

00:03:52.139 --> 00:03:54.433
of x and y that are gonna be at this point

00:03:54.433 --> 00:03:57.615
that maximizes the function
subject to our constraint.

00:03:57.615 --> 00:04:00.443
So that's gonna be related
to the gradient of g,

00:04:00.443 --> 00:04:01.377
it's not gonna be quite equal,

00:04:01.377 --> 00:04:03.006
so I'll leave some room here.

00:04:03.006 --> 00:04:05.041
Related to the gradient of g,

00:04:05.041 --> 00:04:08.124
evaluated at that same point, xm, ym.

00:04:10.972 --> 00:04:13.719
And like I said, they're not
equal, they're proportional.

00:04:13.719 --> 00:04:14.715
So we need to have some kind

00:04:14.715 --> 00:04:16.633
of proportionality constant in there.

00:04:16.633 --> 00:04:19.475
You almost always use the variable lambda,

00:04:19.475 --> 00:04:20.775
and this guy has a fancy name,

00:04:20.775 --> 00:04:24.343
it's called a Lagrange multiplier.

00:04:24.343 --> 00:04:26.284
Lagrange, Lagrange was one

00:04:26.284 --> 00:04:29.418
of those famous French mathematicians.

00:04:29.418 --> 00:04:30.976
I always get him confused
with some of the other

00:04:30.976 --> 00:04:34.814
French mathematicians at the
time like Legendre or Laplace,

00:04:34.814 --> 00:04:36.161
there's a whole bunch of things.

00:04:36.161 --> 00:04:40.513
Let's see, multiplier,
distracting myself talking here.

00:04:40.513 --> 00:04:42.129
So Lagrange multiplier.

00:04:42.129 --> 00:04:44.589
So there's a number of things
in multivariable calculus

00:04:44.589 --> 00:04:47.031
named after Lagrange, and
this is one of the big ones.

00:04:47.031 --> 00:04:50.210
This is a technique that
he kind of developed

00:04:50.210 --> 00:04:52.262
or at the very least popularized.

00:04:52.262 --> 00:04:53.917
And the core idea is to just set

00:04:53.917 --> 00:04:55.721
these gradients equal to each other,

00:04:55.721 --> 00:04:57.860
'cause that represents
when the contour line

00:04:57.860 --> 00:05:02.834
for one function is tangent to
the contour line of another.

00:05:02.834 --> 00:05:05.972
So this, this is something
that we can actually work with.

00:05:05.972 --> 00:05:07.968
And let's start working it out, right,

00:05:07.968 --> 00:05:10.280
let's see what this
translates to in formulas.

00:05:10.280 --> 00:05:11.332
So I already have g written here,

00:05:11.332 --> 00:05:12.776
so let's go ahead and just evaluate

00:05:12.776 --> 00:05:15.526
what the gradient of g should be.

00:05:16.817 --> 00:05:21.184
And that's the gradient of
x squared plus y squared.

00:05:21.184 --> 00:05:23.161
And the way that we take our gradient is

00:05:23.161 --> 00:05:24.913
it's gonna be a vector

00:05:24.913 --> 00:05:26.850
whose components are
all partial derivatives.

00:05:26.850 --> 00:05:28.361
So the first component
is the partial derivative

00:05:28.361 --> 00:05:29.840
with respect to x.

00:05:29.840 --> 00:05:32.701
So we treat x as a variable,
y looks like a constant.

00:05:32.701 --> 00:05:34.325
The derivative is two x.

00:05:34.325 --> 00:05:36.108
The second component
the partial derivative

00:05:36.108 --> 00:05:38.990
with respect to y, so now we're
treating y as the variable,

00:05:38.990 --> 00:05:43.157
x is the constant, so the
derivative looks like two y.

00:05:44.124 --> 00:05:46.674
So that's the gradient of g.

00:05:46.674 --> 00:05:50.590
Then the gradient of f, gradient of f.

00:05:50.590 --> 00:05:53.427
It's gonna look like gradient of,

00:05:53.427 --> 00:05:54.438
let's see, what is x?

00:05:54.438 --> 00:05:55.271
What is f?

00:05:55.271 --> 00:05:57.142
It's x squared times y.

00:05:57.142 --> 00:05:59.142
So x squared times y.

00:06:00.486 --> 00:06:02.131
We do the same thing.

00:06:02.131 --> 00:06:04.604
First component partial
derivative with respect to x,

00:06:04.604 --> 00:06:07.751
x looks like a variable, so
it's derivative is two times x

00:06:07.751 --> 00:06:10.932
and then that y looks like a
constant when we're up here.

00:06:10.932 --> 00:06:13.647
But then partial derivative
with respect to y,

00:06:13.647 --> 00:06:15.928
that y looks like a variable,
that x squared just looks

00:06:15.928 --> 00:06:18.873
like a constant sitting in front of it.

00:06:18.873 --> 00:06:20.277
So that's what we get.

00:06:20.277 --> 00:06:22.307
And now if we kind of work out

00:06:22.307 --> 00:06:24.646
this Lagrange mulitplier expression

00:06:24.646 --> 00:06:29.016
using these two vectors,
what we have written,

00:06:29.016 --> 00:06:32.364
what we're gonna have
written is that this vector,

00:06:32.364 --> 00:06:33.947
two xy x squared

00:06:36.883 --> 00:06:41.050
is proportional with the
proportionality constant lambda to

00:06:43.414 --> 00:06:45.199
the gradient vector for g.

00:06:45.199 --> 00:06:46.699
So two x two y.

00:06:48.673 --> 00:06:49.772
And if you want you can think about this

00:06:49.772 --> 00:06:51.311
as two separate equations.

00:06:51.311 --> 00:06:53.204
I mean right now it's one
equation with vectors,

00:06:53.204 --> 00:06:55.791
but really what this is saying is

00:06:55.791 --> 00:06:57.572
you've got two separate equations,

00:06:57.572 --> 00:07:01.001
two times xy is equal to lambda,

00:07:01.001 --> 00:07:03.049
ah, gotta change colors a lot here.

00:07:03.049 --> 00:07:05.397
Lambda times two x.

00:07:05.397 --> 00:07:08.110
Hm, gonna be stickler for color.

00:07:08.110 --> 00:07:11.787
Keep red all of the
things associated with g.

00:07:11.787 --> 00:07:12.620
And then,

00:07:13.457 --> 00:07:15.728
this second equation is that x squared

00:07:15.728 --> 00:07:18.311
is equal to lambda times two y.

00:07:20.100 --> 00:07:22.906
And this might seem like a problem,

00:07:22.906 --> 00:07:25.177
because we have three unknowns, x, y,

00:07:25.177 --> 00:07:26.911
and this new lambda that we introduced.

00:07:26.911 --> 00:07:27.966
Kind of shot ourselves in the foot

00:07:27.966 --> 00:07:30.325
by giving ourselves a new
variable to deal with.

00:07:30.325 --> 00:07:32.124
But we only have two equations.

00:07:32.124 --> 00:07:35.426
So in order to solve this, we're
gonna need three equations.

00:07:35.426 --> 00:07:36.909
And the third equation is something

00:07:36.909 --> 00:07:38.175
that we've known the whole time.

00:07:38.175 --> 00:07:40.104
It's been part of the original problem.

00:07:40.104 --> 00:07:41.507
It's the constraint itself,

00:07:41.507 --> 00:07:44.507
x squared plus y squared equals one.

00:07:46.138 --> 00:07:48.012
So that, that third equation,

00:07:48.012 --> 00:07:51.921
x squared plus y squared is equal to one.

00:07:51.921 --> 00:07:54.857
So these are the three equations

00:07:54.857 --> 00:07:57.604
that characterize our
constrained optimization problem.

00:07:57.604 --> 00:07:59.079
The bottom one just tells you

00:07:59.079 --> 00:08:02.023
that we have to be on
this unit circle here.

00:08:02.023 --> 00:08:03.558
Allow me to just highlight it.

00:08:03.558 --> 00:08:05.477
We have to be on this unit circle.

00:08:05.477 --> 00:08:08.313
And then these top two
tell us what's necessary

00:08:08.313 --> 00:08:10.025
in order for our contour lines,

00:08:10.025 --> 00:08:12.077
the contour of f and the contour of g

00:08:12.077 --> 00:08:15.133
to be perfectly tangent with each other.

00:08:15.133 --> 00:08:18.391
So, in the next video, I'll
go ahead and solve this.

00:08:18.391 --> 00:08:20.800
At this point, it's pretty
much just algebra to deal with,

00:08:20.800 --> 00:08:22.367
but it's worthy going through.

00:08:22.367 --> 00:08:23.783
And then in the next couple ones,

00:08:23.783 --> 00:08:25.444
I'll talk about a way
that you can encapsulate

00:08:25.444 --> 00:08:28.311
all three of these equations
into one expression,

00:08:28.311 --> 00:08:30.357
and also a little bit
about the interpretation

00:08:30.357 --> 00:08:31.901
of this lambda that we introduced.

00:08:31.901 --> 00:08:33.480
'Cause it's not actually
just a dummy variable,

00:08:33.480 --> 00:08:36.966
it has a pretty interesting
meaning in physical contexts

00:08:36.966 --> 00:08:38.368
once you're actually dealing

00:08:38.368 --> 00:08:41.260
with a constrained optimization
problem in practice.

00:08:41.260 --> 00:08:42.097
So I'll see you in the next video.


Constrained optimization introduction

WEBVTT
Kind: captions
Language: en

00:00:01.023 --> 00:00:01.870
- [Instructor] Hey everyone,

00:00:01.870 --> 00:00:02.834
so in the next couple of videos,

00:00:02.834 --> 00:00:04.153
I'm going to be talking about a different

00:00:04.153 --> 00:00:06.310
sort of optimization problem,

00:00:06.310 --> 00:00:09.268
something called a Constrained
Optimization problem,

00:00:09.268 --> 00:00:12.460
and an example of this is
something where you might see,

00:00:12.460 --> 00:00:15.709
you might be asked to maximize

00:00:15.709 --> 00:00:18.153
some kind of multi-variable function,

00:00:18.153 --> 00:00:20.292
and let's just say it was the function

00:00:20.292 --> 00:00:23.875
f of x,y is equal to x squared, times y.

00:00:26.633 --> 00:00:27.975
But that's not all you're asked to do,

00:00:27.975 --> 00:00:30.021
you're subject to a certain constraint

00:00:30.021 --> 00:00:31.283
where you're only allowed values of

00:00:31.283 --> 00:00:33.366
x and y on a certain set,

00:00:35.087 --> 00:00:36.996
and I'm just going to say,

00:00:36.996 --> 00:00:39.477
the set of all values of x and y,

00:00:39.477 --> 00:00:43.644
such that, x squared plus
y squared, equals one.

00:00:45.851 --> 00:00:47.245
And this is something you might

00:00:47.245 --> 00:00:49.491
recognize as the unit circle,

00:00:49.491 --> 00:00:52.987
this particular constraint
that I've put on here,

00:00:52.987 --> 00:00:55.763
this is the unit circle.

00:00:55.763 --> 00:00:58.210
So one way that you might think
about a problem like this,

00:00:58.210 --> 00:01:01.468
you know, you're maximizing a
certain two-variable function,

00:01:01.468 --> 00:01:03.705
is to first think of the
graph of that function.

00:01:03.705 --> 00:01:05.110
That's what I have pictured here,

00:01:05.110 --> 00:01:09.461
is the graph of f of x,y,
equals x squared, times y.

00:01:09.461 --> 00:01:11.834
And now this constraint,
x squared plus y squared,

00:01:11.834 --> 00:01:14.873
is basically just a
subset of the x,y plane.

00:01:14.873 --> 00:01:16.563
So if we look at it head on here,

00:01:16.563 --> 00:01:18.285
and we look at the x,y plane,

00:01:18.285 --> 00:01:20.009
this circle represents all of

00:01:20.009 --> 00:01:23.176
the points x,y, such that, this holds.

00:01:24.455 --> 00:01:25.480
And what I've actually drawn here

00:01:25.480 --> 00:01:27.113
isn't the circle on the x,y plane,

00:01:27.113 --> 00:01:30.287
but I've projected it up onto the graph.

00:01:30.287 --> 00:01:32.450
So this is showing you
basically the values

00:01:32.450 --> 00:01:34.502
where this constraint holds,

00:01:34.502 --> 00:01:37.969
and also what they look like when graphed.

00:01:37.969 --> 00:01:40.177
So the way you can think
about a problem like this,

00:01:40.177 --> 00:01:42.320
is that you're looking on this circle,

00:01:42.320 --> 00:01:44.781
this kind of projected
circle onto the graph,

00:01:44.781 --> 00:01:46.661
and looking for the highest points.

00:01:46.661 --> 00:01:47.878
And you might notice kind of here,

00:01:47.878 --> 00:01:50.679
there's sort of a peak
on that wiggly circle,

00:01:50.679 --> 00:01:52.268
and over here there's another one,

00:01:52.268 --> 00:01:53.801
and then the low points would be,

00:01:53.801 --> 00:01:57.219
you know, around that
point and around over here.

00:01:57.219 --> 00:01:58.480
Now this is good,

00:01:58.480 --> 00:01:59.869
and I think this a nice way to

00:01:59.869 --> 00:02:02.993
sort of wrap your head around
what this problem is asking,

00:02:02.993 --> 00:02:05.591
but there's actually a
better way to visualize it,

00:02:05.591 --> 00:02:08.814
in terms of finding the actual solution,

00:02:08.814 --> 00:02:11.244
and that's to look only in the x,y plane,

00:02:11.244 --> 00:02:12.973
rather than trying to graph things,

00:02:12.973 --> 00:02:16.253
and just limit our perspective
to the input space.

00:02:16.253 --> 00:02:18.494
So what I have here are the contour lines

00:02:18.494 --> 00:02:21.613
for f of x,y equals x
squared plus y squared.

00:02:21.613 --> 00:02:22.623
And if you're unfamiliar with

00:02:22.623 --> 00:02:24.589
contour lines or a contour map,

00:02:24.589 --> 00:02:25.729
I have a video on that,

00:02:25.729 --> 00:02:27.139
you can go back and take a look,

00:02:27.139 --> 00:02:28.722
it's going to be pretty crucial

00:02:28.722 --> 00:02:31.783
for the next couple videos
to have a feel for that.

00:02:31.783 --> 00:02:34.057
But basically, each one
of these lines represents

00:02:34.057 --> 00:02:36.421
a certain constant value for f.

00:02:36.421 --> 00:02:37.875
So for example, one of them,

00:02:37.875 --> 00:02:41.858
one of them might represent
all of the values of x and y,

00:02:41.858 --> 00:02:45.912
where f of x,y is equal to, you know, two,

00:02:45.912 --> 00:02:47.675
right, so if you looked
at all of the values

00:02:47.675 --> 00:02:49.266
of x and y where this is true,

00:02:49.266 --> 00:02:51.661
you'd find yourself on one of these lines,

00:02:51.661 --> 00:02:53.838
and each line represents
a different possible value

00:02:53.838 --> 00:02:57.171
for what this constant here actually is.

00:02:58.275 --> 00:02:59.108
So what I'm going to do here,

00:02:59.108 --> 00:03:00.671
is I'm actually going to just zoom in

00:03:00.671 --> 00:03:04.085
on one particular contour line, right.

00:03:04.085 --> 00:03:05.683
So this here is something
that I'm going to vary,

00:03:05.683 --> 00:03:07.358
where I'm going to be able to change

00:03:07.358 --> 00:03:10.328
what the constant we're
setting f equal to is,

00:03:10.328 --> 00:03:13.053
and look at how the contour
line changes as a result.

00:03:13.053 --> 00:03:16.040
So for example, if I
put it around here-ish,

00:03:16.040 --> 00:03:18.650
what you're looking at
is the contour line,

00:03:18.650 --> 00:03:22.067
the contour line for f of x,y equals 0.1.

00:03:24.202 --> 00:03:26.816
So all of the values on
these two blue lines here

00:03:26.816 --> 00:03:30.733
tell you what values
of x and y satisfy 0.1.

00:03:31.777 --> 00:03:34.708
But on the other hand, I
could also shift this guy up,

00:03:34.708 --> 00:03:36.170
and maybe I'll shift it up,

00:03:36.170 --> 00:03:37.442
I'm going to set to where that

00:03:37.442 --> 00:03:39.929
constant is actually equal to one,

00:03:39.929 --> 00:03:41.607
so that would be kind of an alternative,

00:03:41.607 --> 00:03:44.090
we'll just kind of separate over here.

00:03:44.090 --> 00:03:47.173
That would be the line where f of x,y

00:03:48.647 --> 00:03:51.303
is set equal to one, itself.

00:03:51.303 --> 00:03:52.921
And the main thing I
want to highlight here,

00:03:52.921 --> 00:03:56.094
is that at some values, like 0.1,

00:03:56.094 --> 00:03:59.004
this contour line
intersects with the circle,

00:03:59.004 --> 00:04:00.898
it intersects with our constraint.

00:04:00.898 --> 00:04:04.914
And let's just thing
about what that means,

00:04:04.914 --> 00:04:06.855
if there's a point, x and y,

00:04:06.855 --> 00:04:08.484
on that intersection there,

00:04:08.484 --> 00:04:12.031
that basically gives us a
pair of numbers, x and y,

00:04:12.031 --> 00:04:14.782
such that, this is true,

00:04:14.782 --> 00:04:17.499
that fact that f of x,y equals 0.1,

00:04:17.499 --> 00:04:21.666
and also that x squared
plus y squared equals one.

00:04:22.806 --> 00:04:23.910
So it means this is something that

00:04:23.910 --> 00:04:26.098
actually exists and is possible.

00:04:26.098 --> 00:04:27.625
And in fact, we can see
that there is four different

00:04:27.625 --> 00:04:29.883
pairs of numbers where that's true,

00:04:29.883 --> 00:04:30.884
where they intersect here,

00:04:30.884 --> 00:04:32.510
where they intersect over here,

00:04:32.510 --> 00:04:33.382
and then the other two,

00:04:33.382 --> 00:04:35.428
kind of symmetrically on that side.

00:04:35.428 --> 00:04:37.599
But on the other hand, if
we look as this other world,

00:04:37.599 --> 00:04:41.766
where we shift up to the
line f of x,y equals one,

00:04:43.125 --> 00:04:45.841
this never intersects with the constraint.

00:04:45.841 --> 00:04:48.008
So what that means is x,y,

00:04:50.683 --> 00:04:53.751
the pairs of numbers
that satisfy this guy,

00:04:53.751 --> 00:04:55.684
are off the constraint,

00:04:55.684 --> 00:04:57.918
they're off of that circle,

00:04:57.918 --> 00:05:00.918
x squared plus y squared equals one.

00:05:02.212 --> 00:05:03.201
So what that tells us,

00:05:03.201 --> 00:05:05.749
as we try to maximize this function,

00:05:05.749 --> 00:05:07.344
subject to this constraint,

00:05:07.344 --> 00:05:10.030
is that we can never get as high as one.

00:05:10.030 --> 00:05:12.700
0.1 would be achievable, and in fact,

00:05:12.700 --> 00:05:17.684
if we kind of go back to
that, and we look at 0.1,

00:05:17.684 --> 00:05:19.121
if I upped that value,

00:05:19.121 --> 00:05:21.168
and you know, changed
it to the line where,

00:05:21.168 --> 00:05:24.665
instead what you're looking at is 0.2,

00:05:24.665 --> 00:05:25.498
that's also possible,

00:05:25.498 --> 00:05:27.462
because it intersects with the circle.

00:05:27.462 --> 00:05:29.000
And in fact, you could
play around with it,

00:05:29.000 --> 00:05:31.204
and increase it a little bit more,

00:05:31.204 --> 00:05:33.790
and if I go to 0.3, instead,

00:05:33.790 --> 00:05:38.460
and I go over here and I say,
0.3, that's also possible.

00:05:38.460 --> 00:05:39.979
And what we're basically trying to do

00:05:39.979 --> 00:05:42.775
is find the maximum value
that we can put here,

00:05:42.775 --> 00:05:44.538
the maximum value so that if we look at

00:05:44.538 --> 00:05:48.430
the line that represents f
of x,y equals that value,

00:05:48.430 --> 00:05:52.055
it still intersects with the circle.

00:05:52.055 --> 00:05:54.545
And the key here, the key observation,

00:05:54.545 --> 00:05:56.347
is that that maximum value happens

00:05:56.347 --> 00:05:58.680
when these guys are tangent.

00:06:00.065 --> 00:06:00.898
And in the next video,

00:06:00.898 --> 00:06:02.187
I'll start going into the details

00:06:02.187 --> 00:06:03.998
of how we can use that observation,

00:06:03.998 --> 00:06:06.810
this notion of tangency,
to solve the problem,

00:06:06.810 --> 00:06:08.956
to find the actual value of x and y

00:06:08.956 --> 00:06:11.929
that maximizes this,
subject to the constraint.

00:06:11.929 --> 00:06:13.817
But in the interim, I kind
of want you to mull on that,

00:06:13.817 --> 00:06:15.951
and think a little bit about
how you might use that.

00:06:15.951 --> 00:06:18.067
What does tangency mean here?

00:06:18.067 --> 00:06:19.918
How can you take advantage
of certain other notions

00:06:19.918 --> 00:06:22.084
that we've learned about
in multi-variable calculus,

00:06:22.084 --> 00:06:23.885
like, hint, hint, the gradient,

00:06:23.885 --> 00:06:25.851
to actually solve something like this.

00:06:25.851 --> 00:06:28.326
So with that, I will see you next video.


Matched pairs experiment design

WEBVTT
Kind: captions
Language: en

00:00:00.148 --> 00:00:01.681
- [Narrator] The last video we constructed

00:00:01.681 --> 00:00:04.792
an experiment where we
had a drug that we thought

00:00:04.792 --> 00:00:07.834
might help control people's blood sugar.

00:00:07.834 --> 00:00:10.249
We looked for something
that we could measure

00:00:10.249 --> 00:00:12.803
as an indicator for their
blood sugar's being controlled,

00:00:12.803 --> 00:00:15.497
and hemoglobin A1c is
actually what people measure

00:00:15.497 --> 00:00:16.797
in a blood test.

00:00:16.797 --> 00:00:19.467
It is, we have a whole
video on it on Khan Academy.

00:00:19.467 --> 00:00:22.114
But it is an average
measure of your blood sugar

00:00:22.114 --> 00:00:24.274
over roughly a three month period.

00:00:24.274 --> 00:00:26.015
So that's the explanatory variable.

00:00:26.015 --> 00:00:27.594
Whether or not you're taking the pill.

00:00:27.594 --> 00:00:29.452
And the response variable
is well, what does it do

00:00:29.452 --> 00:00:30.984
to your hemoglobin A1c.

00:00:30.984 --> 00:00:33.283
And we constructed a
somewhat classic experiment

00:00:33.283 --> 00:00:35.257
where we had a control
group and a treatment group,

00:00:35.257 --> 00:00:37.486
and we randomly assigned
folks to either the control

00:00:37.486 --> 00:00:40.017
or the treatment group.

00:00:40.017 --> 00:00:42.618
And to ensure that one group of the other,

00:00:42.618 --> 00:00:45.079
or I guess both of them, don't
end up with an imbalance of,

00:00:45.079 --> 00:00:48.887
in the case, in the
case of the last video,

00:00:48.887 --> 00:00:52.085
an imbalance of men or
women, we did what we call

00:00:52.085 --> 00:00:54.663
a block design, where
we took our 100 people,

00:00:54.663 --> 00:00:57.797
and we just happened to
have 60 women and 40 men.

00:00:57.797 --> 00:01:00.886
We said, okay, let's split
the 60 women randomly

00:01:00.886 --> 00:01:03.765
between the two groups,
and let's split the 40 men

00:01:03.765 --> 00:01:07.201
between these two groups,
so that we have at least

00:01:07.201 --> 00:01:10.661
an even distribution with respect to sex.

00:01:10.661 --> 00:01:13.418
And we would measure folks'
A1c's before they get

00:01:13.418 --> 00:01:15.578
the treatment or the placebo.

00:01:15.578 --> 00:01:18.132
Then we would wait three months of getting

00:01:18.132 --> 00:01:20.361
either the treatment or the placebo.

00:01:20.361 --> 00:01:22.474
And then we'll see if
there is a statistically

00:01:22.474 --> 00:01:23.983
significant improvement.

00:01:23.983 --> 00:01:26.491
Now this was a pretty good,
and it's a bit of a classic

00:01:26.491 --> 00:01:28.441
experimental design.

00:01:28.441 --> 00:01:30.137
We would also do it so that
the patients don't know

00:01:30.137 --> 00:01:33.434
which one they're getting,
placebo or the actual treatment,

00:01:33.434 --> 00:01:34.989
so it's a blind experiment.

00:01:34.989 --> 00:01:36.963
And it'll probably be
good if even the nurses

00:01:36.963 --> 00:01:38.496
or the doctors who are
administering the pills

00:01:38.496 --> 00:01:40.516
or giving the pills also don't know which

00:01:40.516 --> 00:01:41.349
one they're getting.

00:01:41.349 --> 00:01:42.931
So it would be a doubles blind experiment.

00:01:42.931 --> 00:01:44.858
But this doesn't mean that
it's a perfect experiment.

00:01:44.858 --> 00:01:46.808
There seldom is a perfect experiment,

00:01:46.808 --> 00:01:49.270
and that's why it should
be able to be replicated,

00:01:49.270 --> 00:01:51.824
that other people should
try to prove the same thing.

00:01:51.824 --> 00:01:53.937
It may be in different ways,

00:01:53.937 --> 00:01:55.144
But even the way that we designed it,

00:01:55.144 --> 00:01:58.299
There's still a possibility
that there's some

00:01:58.299 --> 00:01:59.364
lurking variables in here.

00:01:59.364 --> 00:02:03.730
Maybe we took care to
make sure our distribution

00:02:03.730 --> 00:02:06.075
of men and women was roughly even

00:02:06.075 --> 00:02:07.979
across both of these groups,
but maybe my throwing

00:02:07.979 --> 00:02:11.857
at random sampling, we got
a disproportionate number

00:02:11.857 --> 00:02:14.411
of young people and the treatment group.

00:02:14.411 --> 00:02:18.358
And maybe young people responded
better to taking a pill.

00:02:18.358 --> 00:02:21.655
Maybe that changes their
behaviors in other ways.

00:02:21.655 --> 00:02:24.953
But maybe older people,
when they take a pill,

00:02:24.953 --> 00:02:25.786
they decide to eat worse
because they say the pills

00:02:25.786 --> 00:02:26.784
are gonna solve all my problems.

00:02:26.784 --> 00:02:29.149
And so you can have these
other lurking variables

00:02:29.149 --> 00:02:32.818
like age, or where in
the country the live,

00:02:32.818 --> 00:02:33.863
or other types of things, that
just by the random process

00:02:33.863 --> 00:02:36.530
you might have things get uneven

00:02:37.572 --> 00:02:39.290
one way or another.

00:02:39.290 --> 00:02:40.962
Now one technique

00:02:40.962 --> 00:02:44.445
To help control for this a little bit.

00:02:44.445 --> 00:02:46.279
And I shouldn't use controlled too much.

00:02:46.279 --> 00:02:47.789
Another technique to help mitigate this,

00:02:47.789 --> 00:02:51.206
is something called matched pairs design.

00:02:52.061 --> 00:02:53.978
Matched, matched pairs.

00:02:56.612 --> 00:03:00.397
Pairs design of an experiment,
and it's essentially

00:03:00.397 --> 00:03:04.437
instead of going through
all this trouble saying,

00:03:04.437 --> 00:03:05.436
oh boy maybe we do block design

00:03:05.436 --> 00:03:06.876
or all this random sampling.

00:03:06.876 --> 00:03:10.614
Instead, you randomly put
people first into either

00:03:10.614 --> 00:03:12.170
the control or the treatment group,

00:03:12.170 --> 00:03:14.585
and then we do another round you measure,

00:03:14.585 --> 00:03:16.326
and then you do another
round where you switch,

00:03:16.326 --> 00:03:17.998
where the people who are in the treatment

00:03:17.998 --> 00:03:19.693
go to the control, and the
people who are in the control

00:03:19.693 --> 00:03:20.715
go into the treatment.

00:03:20.715 --> 00:03:22.294
So we can even extend
from what we have here.

00:03:22.294 --> 00:03:26.461
We can imagine a a world
where the first three months

00:03:28.749 --> 00:03:32.139
we have the 50 people
in this treatment group,

00:03:32.139 --> 00:03:34.298
we have another 50 people
in this control group,

00:03:34.298 --> 00:03:36.086
they are taking the placebo.

00:03:36.086 --> 00:03:39.198
We see what happens to the
A1Cs, and then we switch.

00:03:39.198 --> 00:03:41.448
Where this group over here.

00:03:43.075 --> 00:03:45.351
Then, they don't know, they don't know.

00:03:45.351 --> 00:03:47.209
First of all, ideally
it's a blind experiment

00:03:47.209 --> 00:03:48.950
so they don't even know
they're on the treatment group,

00:03:48.950 --> 00:03:51.295
so hopefully the pills look identical.

00:03:51.295 --> 00:03:55.661
So now that same group
for the next three months

00:03:55.661 --> 00:03:58.261
is now going to be the control group.

00:03:58.261 --> 00:04:01.350
And so they got the medicine
for the first there months

00:04:01.350 --> 00:04:03.138
and we saw what happens to their A1c,

00:04:03.138 --> 00:04:04.995
and now we're gonna pet the placebo,

00:04:04.995 --> 00:04:06.713
then we're going to get the placebo

00:04:06.713 --> 00:04:08.223
for the second three months,

00:04:08.223 --> 00:04:12.077
And then we're going to
see what happens to their

00:04:12.077 --> 00:04:15.862
A1c and likewise the
other group is going to be

00:04:15.862 --> 00:04:16.791
switched around.

00:04:16.791 --> 00:04:18.974
The thing that the folks
that used to be getting

00:04:18.974 --> 00:04:22.307
the placebo could now get the treatment.

00:04:25.074 --> 00:04:27.861
They are now going to get the treatment.

00:04:27.861 --> 00:04:30.972
And the value here is,
is that because everyone

00:04:30.972 --> 00:04:33.991
is going through, is it for one period

00:04:33.991 --> 00:04:36.800
in the control group, and once period

00:04:36.800 --> 00:04:37.799
is in the treatment group.

00:04:37.799 --> 00:04:39.540
But they don't know
when, which one happening

00:04:39.540 --> 00:04:42.466
you are less likely to
have a lurking variable

00:04:42.466 --> 00:04:44.883
like age or geographic region

00:04:45.879 --> 00:04:48.387
or behavior, cause an imbalance

00:04:48.387 --> 00:04:51.359
or in somehow skew the results

00:04:51.359 --> 00:04:54.192
of give you a biased result.

00:04:54.192 --> 00:04:55.817
So this is an interesting thing,

00:04:55.817 --> 00:04:57.512
even when I've talked
about it in this video

00:04:57.512 --> 00:04:58.999
in the last one, these aren't just

00:04:58.999 --> 00:05:00.763
different ways to approach it,

00:05:00.763 --> 00:05:02.156
and as you construct experiments,

00:05:02.156 --> 00:05:03.062
this is in medicine.

00:05:03.062 --> 00:05:04.455
You'll obviously construct experiments

00:05:04.455 --> 00:05:05.788
in other fields.

00:05:07.009 --> 00:05:10.214
It's important to think
about what types of things

00:05:10.214 --> 00:05:11.047
are practical to do.

00:05:11.047 --> 00:05:13.952
And also (stammering) have the best chance

00:05:13.952 --> 00:05:17.087
at giving you real, I guess you could say,

00:05:17.087 --> 00:05:19.754
an unbiased and real information

00:05:21.127 --> 00:05:22.892
as to in the case of an experiment

00:05:22.892 --> 00:05:26.677
to the efficacy of something,
or whether a certain variable,

00:05:26.677 --> 00:05:30.177
an explanatory variable really does drive,

00:05:31.414 --> 00:05:35.164
have a causal effect
on a response variable.


Introduction to experiment design

WEBVTT
Kind: captions
Language: en

00:00:00.544 --> 00:00:03.299
- [Instructor] So let's say
that I am a drug company

00:00:03.299 --> 00:00:05.687
and I have come up with
a medicine that I think

00:00:05.687 --> 00:00:09.047
will help folks with
diabetes, and in particular,

00:00:09.047 --> 00:00:13.802
I think it will help reduce
their hemoglobin A1c levels,

00:00:13.802 --> 00:00:16.178
and for those of you who aren't familiar

00:00:16.178 --> 00:00:18.635
with what hemoglobin
A1c is, I encourage you,

00:00:18.635 --> 00:00:20.745
we have a video on that on Khan Academy,

00:00:20.745 --> 00:00:23.240
but the general idea is if
you have high blood sugar

00:00:23.240 --> 00:00:26.392
over roughly a three-month period of time,

00:00:26.392 --> 00:00:30.559
high blood sugar, and I can
say high average blood sugar,

00:00:31.463 --> 00:00:35.630
you're going to have a high
A1c, a high hemoglobin A1c level

00:00:36.471 --> 00:00:40.588
and if you have a low average blood sugar

00:00:40.588 --> 00:00:42.478
over roughly a three-month
time, you're going

00:00:42.478 --> 00:00:45.061
to have a lower hemoglobin A1c.

00:00:46.248 --> 00:00:49.652
So if taking the pill
seems to lower folks'

00:00:49.652 --> 00:00:53.207
A1c levels more than is likely to happen

00:00:53.207 --> 00:00:56.034
due to randomly or due to other variables,

00:00:56.034 --> 00:00:57.742
well then that means that your new pill

00:00:57.742 --> 00:01:02.645
might be effective at
controlling folks' diabetes.

00:01:02.645 --> 00:01:06.522
So in this situation, when
we're constructing an experiment

00:01:06.522 --> 00:01:09.970
to test this, we would
say that whether or not

00:01:09.970 --> 00:01:14.483
you are taking the pill, this
is the explanatory variable.

00:01:14.483 --> 00:01:18.650
Explanatory variable, and the
thing that it is affecting,

00:01:20.281 --> 00:01:22.663
the thing that you're
hoping has some response,

00:01:22.663 --> 00:01:25.806
in this case the A1c
levels are your indicator

00:01:25.806 --> 00:01:28.228
of whether it is help
controlling the blood sugar,

00:01:28.228 --> 00:01:31.145
we call that the response variable.

00:01:32.125 --> 00:01:36.042
That right over there is
the response variable.

00:01:39.066 --> 00:01:43.847
So how are we actually going
to conduct this experiment?

00:01:43.847 --> 00:01:47.313
Well let's say that we
have a group of folks,

00:01:47.313 --> 00:01:51.419
let's say that we have been
given a group of 100 folks

00:01:51.419 --> 00:01:54.333
who need to control their diabetes.

00:01:54.333 --> 00:01:58.500
So 100 people here who need
to control their diabetes,

00:01:59.521 --> 00:02:03.471
and we say, "Alright, well
let's take half of this group

00:02:03.471 --> 00:02:06.862
"and put them into, I guess
you could say a treatment group

00:02:06.862 --> 00:02:10.609
"and another half and put
them into a control group

00:02:10.609 --> 00:02:12.802
"and see if the treatment
group, the one that actually

00:02:12.802 --> 00:02:16.969
"gets my pill is going to
improve their A1c levels in a way

00:02:18.681 --> 00:02:21.854
"that seems like it would
not be just random chance."

00:02:21.854 --> 00:02:26.129
So let's do that, so we're
going to have a control group,

00:02:26.129 --> 00:02:29.962
so this is my control
group, control, and this

00:02:32.265 --> 00:02:36.432
is the treatment group,
this is the treatment group.

00:02:40.833 --> 00:02:43.263
And you might say, "Okay,
we'll just give these folks,

00:02:43.263 --> 00:02:45.773
"the treatment group the
pill and then we won't give

00:02:45.773 --> 00:02:49.600
"the pill that I created
to the control group."

00:02:49.600 --> 00:02:53.005
But that might introduce
a psychological aspect

00:02:53.005 --> 00:02:56.705
that maybe the benefit of the
pill is just people feeling,

00:02:56.705 --> 00:02:59.210
"Hey I'm taking something
that'll control my diabetes,"

00:02:59.210 --> 00:03:01.760
maybe that psychologically
affects their blood sugar

00:03:01.760 --> 00:03:04.484
in some way and this is actually possible,

00:03:04.484 --> 00:03:07.279
maybe it makes them act
healthier in certain ways,

00:03:07.279 --> 00:03:09.342
maybe that makes them act
unhealthier in certain ways

00:03:09.342 --> 00:03:11.478
'cause they're like, "Oh
I have a pill to control

00:03:11.478 --> 00:03:14.087
"my diabetes, my blood sugar, I can go eat

00:03:14.087 --> 00:03:16.337
"more sweets now and it'll control it."

00:03:16.337 --> 00:03:19.756
And so to avoid that, in
order for just the very fact

00:03:19.756 --> 00:03:22.264
that someone says, "Hey I
think I'm taking a medicine,

00:03:22.264 --> 00:03:24.436
"I might behave in a
different way or it might even

00:03:24.436 --> 00:03:28.080
"psychologically affect
my body in a certain way,"

00:03:28.080 --> 00:03:31.270
what we wanna do is
give both groups a pill,

00:03:31.270 --> 00:03:33.800
and we wanna do it in a
way that neither group

00:03:33.800 --> 00:03:36.553
knows which pill they're getting.

00:03:36.553 --> 00:03:39.683
So what we would do here
is we would give this group

00:03:39.683 --> 00:03:42.683
a placebo, a placebo, and this group

00:03:44.845 --> 00:03:49.150
would actually get the
medicine, the medicine,

00:03:49.150 --> 00:03:52.427
but those pills should look
the same, and people should

00:03:52.427 --> 00:03:56.026
not know which group they
are in and that is a,

00:03:56.026 --> 00:04:00.193
when we do that, that is a
blind experiment, experiment.

00:04:02.301 --> 00:04:05.879
Now you might have heard about
double-blind experiments.

00:04:05.879 --> 00:04:09.390
Well that would be the case
where not only do people

00:04:09.390 --> 00:04:12.317
not know which group
they're in, but even their

00:04:12.317 --> 00:04:15.470
physician or the person who's
administering the experiment,

00:04:15.470 --> 00:04:17.841
they don't know which one they're giving,

00:04:17.841 --> 00:04:19.739
they don't know if
they're giving the placebo

00:04:19.739 --> 00:04:23.144
or the actual medicine to the group.

00:04:23.144 --> 00:04:24.720
So let's say we wanna do that.

00:04:24.720 --> 00:04:29.071
So we could do double,
double-blind experiment,

00:04:29.071 --> 00:04:30.699
so even the person giving the pill

00:04:30.699 --> 00:04:32.471
doesn't know which pill they're giving.

00:04:32.471 --> 00:04:34.307
And you might say, "Well
why is that important?"

00:04:34.307 --> 00:04:38.988
Well if the physician knows,
or the person administering

00:04:38.988 --> 00:04:42.230
or interfacing with the
patient, they might give a tell

00:04:42.230 --> 00:04:45.372
somehow, they might not
put as much emphasis

00:04:45.372 --> 00:04:47.779
on the importance of taking
the pill if it's a placebo,

00:04:47.779 --> 00:04:51.820
they might by accident give
away some type of information.

00:04:51.820 --> 00:04:54.319
So to avoid that type of thing happening,

00:04:54.319 --> 00:04:57.288
you could do a double-blind,
and there's even,

00:04:57.288 --> 00:04:59.468
some people talk about a
triple-blind experiment

00:04:59.468 --> 00:05:01.076
where even the people analyzing the data

00:05:01.076 --> 00:05:03.511
don't know which group
was the control group

00:05:03.511 --> 00:05:05.178
and which group was the treatment group,

00:05:05.178 --> 00:05:09.281
and once again, that's
another way to avoid bias.

00:05:09.281 --> 00:05:11.177
So now that we've kinda
figured out, we have a control

00:05:11.177 --> 00:05:14.303
group, we have a treatment
group, we're using A1c

00:05:14.303 --> 00:05:17.433
as our response variable, so
we would wanna measure folks'

00:05:17.433 --> 00:05:22.188
A1c levels, their hemoglobin
A1c levels before they get

00:05:22.188 --> 00:05:24.479
either the placebo or the medicine

00:05:24.479 --> 00:05:26.770
and then maybe after three months,

00:05:26.770 --> 00:05:29.520
we would measure their A1c after,

00:05:30.359 --> 00:05:32.873
but the next question is, how
do you divvy these 100 people

00:05:32.873 --> 00:05:36.379
up into these two groups,
and you might say,

00:05:36.379 --> 00:05:39.035
"Well I would wanna do it
randomly," and you would be right

00:05:39.035 --> 00:05:40.486
'cause if you didn't do
it randomly, if you put

00:05:40.486 --> 00:05:42.137
all the men here and all the women here,

00:05:42.137 --> 00:05:46.242
well that might, first of
all, sex might explain it

00:05:46.242 --> 00:05:49.992
or behavior of men versus
women might explain

00:05:52.296 --> 00:05:54.893
the differences or the
non-differences you see

00:05:54.893 --> 00:05:58.197
in A1c level, if you get
a lot of people of one age

00:05:58.197 --> 00:06:00.862
or one part of the country
or one type of dietary habit,

00:06:00.862 --> 00:06:04.152
you don't want that, so in order to avoid

00:06:04.152 --> 00:06:08.020
having an imbalance of some
of those lurking variables,

00:06:08.020 --> 00:06:10.978
you would want to randomly
sample and we've done multiple

00:06:10.978 --> 00:06:14.478
videos already on ways to randomly sample,

00:06:15.897 --> 00:06:17.851
so you're going to randomly sample

00:06:17.851 --> 00:06:20.069
and put people into either groups.

00:06:20.069 --> 00:06:23.748
And a very simple way of
doing that, you could give

00:06:23.748 --> 00:06:25.893
everyone here a number from one to 100,

00:06:25.893 --> 00:06:29.611
use a random number generator to do that

00:06:29.611 --> 00:06:33.303
and then, or you could use
a random number generator,

00:06:33.303 --> 00:06:35.738
pick 50 names to put in the control group

00:06:35.738 --> 00:06:37.509
or 50 names to put in the treatment group

00:06:37.509 --> 00:06:40.180
and then everyone else gets
put in the other group.

00:06:40.180 --> 00:06:43.940
Now, to avoid a situation,
just randomly by doing

00:06:43.940 --> 00:06:47.482
a random sample, you might
have a situation where

00:06:47.482 --> 00:06:50.125
there's some probability
that you disproportionately

00:06:50.125 --> 00:06:53.483
have more men in one group or
more women in another group

00:06:53.483 --> 00:06:55.793
and to avoid that, you
could do really a version

00:06:55.793 --> 00:06:57.627
of stratified sampling
that we've talked about

00:06:57.627 --> 00:07:00.891
in other videos, which is
you could do what's called

00:07:00.891 --> 00:07:03.211
a block design for your random assignment

00:07:03.211 --> 00:07:07.011
where you actually split
everyone into men and women

00:07:07.011 --> 00:07:08.835
and it might be 50-50 or it might even be

00:07:08.835 --> 00:07:13.002
just randomly here you got
60 women, 60 women and 40 men

00:07:17.336 --> 00:07:19.994
and what you do here is you
say, "Okay let's randomly

00:07:19.994 --> 00:07:22.798
"take 30 of these women and
put 'em in the control group

00:07:22.798 --> 00:07:25.122
"and 30 of the women and put
'em in the treatment group

00:07:25.122 --> 00:07:28.091
"and let's put randomly 20 of
the men in the control group

00:07:28.091 --> 00:07:29.990
"and 20 of the men in the treatment group"

00:07:29.990 --> 00:07:33.599
and that way someone's sex is less likely

00:07:33.599 --> 00:07:37.766
to introduce bias into
what actually happens here,

00:07:38.959 --> 00:07:43.680
so once again, doing this
is called a block design,

00:07:43.680 --> 00:07:46.283
really a version of stratified sampling.

00:07:46.283 --> 00:07:49.350
Block design, and there might
be other lurking variables

00:07:49.350 --> 00:07:51.635
that you wanna make sure
doesn't just show up here

00:07:51.635 --> 00:07:53.889
randomly and so you might want,

00:07:53.889 --> 00:07:57.306
there's other ways of randomly assigning.

00:07:58.194 --> 00:08:02.616
Now once you do this, you
see what was a change in A1c.

00:08:02.616 --> 00:08:06.383
If you see that, hey, the change in A1c,

00:08:06.383 --> 00:08:09.309
one if you see there's no
difference in A1c levels

00:08:09.309 --> 00:08:10.586
between these two groups, and you're like,

00:08:10.586 --> 00:08:13.524
"Hey, there's a good probability
that my pill does nothing"

00:08:13.524 --> 00:08:15.919
and once again it's all
about probabilities,

00:08:15.919 --> 00:08:17.982
there's some chance
that you're just unlucky

00:08:17.982 --> 00:08:19.562
and it might be a very
small chance and that's why

00:08:19.562 --> 00:08:21.227
you wanna do this with
a good number of people

00:08:21.227 --> 00:08:23.839
and as we forward our
statistics understandings,

00:08:23.839 --> 00:08:26.913
we will better understand
at what threshold levels

00:08:26.913 --> 00:08:28.845
do we think the probability
is high or low enough

00:08:28.845 --> 00:08:32.124
for us to really feel
good about our findings.

00:08:32.124 --> 00:08:35.874
But let's say that you
do see an improvement,

00:08:37.299 --> 00:08:40.695
you need to think about,
is that improvement,

00:08:40.695 --> 00:08:43.104
could that have happened
due to random chance

00:08:43.104 --> 00:08:45.100
or is it very unlikely that that happened

00:08:45.100 --> 00:08:47.945
due purely to random chance,
and if it was very unlikely

00:08:47.945 --> 00:08:50.802
that it happened due
purely to random chance,

00:08:50.802 --> 00:08:52.383
then you would feel pretty good,

00:08:52.383 --> 00:08:53.777
and other people when
you publish the results,

00:08:53.777 --> 00:08:56.495
would feel pretty good
about your medicine.

00:08:56.495 --> 00:09:00.196
Now, even then, science is not done.

00:09:00.196 --> 00:09:02.086
No one will say that they are 100% sure

00:09:02.086 --> 00:09:04.825
that your medicine is good,
there still might have been

00:09:04.825 --> 00:09:06.554
some lurking variables that we
did not, that our experiment

00:09:06.554 --> 00:09:10.387
did not properly adjust
for, that just when we

00:09:12.847 --> 00:09:14.833
even did this block design, we might

00:09:14.833 --> 00:09:17.921
have disproportionately
gotten randomly older people

00:09:17.921 --> 00:09:19.701
in one of the groups
or the other or people

00:09:19.701 --> 00:09:22.220
from one part of the country
in one group or another

00:09:22.220 --> 00:09:24.165
so there's always things to think about

00:09:24.165 --> 00:09:25.638
and the most important
thing to think about,

00:09:25.638 --> 00:09:28.701
even if you did this as good as you could,

00:09:28.701 --> 00:09:31.943
you still, some random
chance might have given you

00:09:31.943 --> 00:09:36.412
a false positive, you got
good results even though

00:09:36.412 --> 00:09:39.088
it was random, or a false
negative, you got bad results

00:09:39.088 --> 00:09:42.005
even though it was actually random.

00:09:43.506 --> 00:09:46.703
And so a very important
idea in experiments

00:09:46.703 --> 00:09:50.458
and this is in science in
general is that this experiment,

00:09:50.458 --> 00:09:54.208
you should document it
well and it should be,

00:09:57.963 --> 00:10:00.370
the process of replication,
other people should

00:10:00.370 --> 00:10:03.006
be able to replicate this experiment

00:10:03.006 --> 00:10:05.807
and hopefully get consistent results

00:10:05.807 --> 00:10:07.125
so it's not just about the results,

00:10:07.125 --> 00:10:10.167
it's your experiment
design, other people should,

00:10:10.167 --> 00:10:13.426
it should be an experiment
that other people could

00:10:13.426 --> 00:10:16.978
and should replicate to reinforce the idea

00:10:16.978 --> 00:10:19.935
that your results are actually true

00:10:19.935 --> 00:10:21.731
and not just random or just due to some

00:10:21.731 --> 00:10:25.398
bad administration of
the actual experiment.


Judging outliers in a dataset

WEBVTT
Kind: captions
Language: en

00:00:00.000 --> 00:00:02.775
- [Instructor] We have a
list of 15 numbers here,

00:00:02.775 --> 00:00:05.634
and what I want to do is
think about the outliers.

00:00:05.634 --> 00:00:09.502
And to help us with that,
let's actually visualize this,

00:00:09.502 --> 00:00:12.318
the distribution of actual numbers.

00:00:12.318 --> 00:00:13.818
So let us do that.

00:00:14.887 --> 00:00:16.242
So here, on a number line,

00:00:16.242 --> 00:00:19.409
I have all the numbers from one to 19.

00:00:20.494 --> 00:00:23.540
And let's see, we have two ones.

00:00:23.540 --> 00:00:27.666
So I could say that's one
one and then two ones.

00:00:27.666 --> 00:00:29.403
We have one six.

00:00:29.403 --> 00:00:31.681
So let's put that six there.

00:00:31.681 --> 00:00:33.098
We have got a 13,

00:00:34.531 --> 00:00:36.009
or we have two 13s.

00:00:36.009 --> 00:00:39.842
So we're gonna go up
here, one 13 and two 13s.

00:00:41.367 --> 00:00:43.784
Let's see, we have three 14s.

00:00:45.105 --> 00:00:45.938
So 14,

00:00:46.962 --> 00:00:48.258
14,

00:00:48.258 --> 00:00:49.091
and 14.

00:00:50.153 --> 00:00:53.050
We have a couple of 15s, 15, 15.

00:00:53.050 --> 00:00:53.883
So 15,

00:00:55.148 --> 00:00:56.444
15.

00:00:56.444 --> 00:00:58.471
We have one 16.

00:00:58.471 --> 00:01:01.103
So that's our 16 there.

00:01:01.103 --> 00:01:03.128
We have three 18s.

00:01:03.128 --> 00:01:04.725
One, two, three.

00:01:04.725 --> 00:01:05.558
So one,

00:01:06.680 --> 00:01:08.085
two,

00:01:08.085 --> 00:01:09.996
and then three.

00:01:09.996 --> 00:01:12.575
And then we have a 19.

00:01:12.575 --> 00:01:14.590
Then we have a 19.

00:01:14.590 --> 00:01:15.667
So when you look,

00:01:15.667 --> 00:01:17.819
when you look visually at
the distribution of numbers,

00:01:17.819 --> 00:01:20.824
it looks like the meat of the
distribution, so to speak,

00:01:20.824 --> 00:01:23.757
is in this area, right over here.

00:01:23.757 --> 00:01:25.012
And so some people might say,

00:01:25.012 --> 00:01:26.610
"Okay, we have three outliers.

00:01:26.610 --> 00:01:28.411
"There are these two ones and the six."

00:01:28.411 --> 00:01:29.244
Some people might say,

00:01:29.244 --> 00:01:31.333
"Well, the six is kinda close enough.

00:01:31.333 --> 00:01:33.889
"Maybe only these two ones are outliers."

00:01:33.889 --> 00:01:37.758
And those would actually be
both reasonable things to say.

00:01:37.758 --> 00:01:40.849
Now to get on the same page,

00:01:40.849 --> 00:01:44.829
statisticians will use a rule sometimes.

00:01:44.829 --> 00:01:46.991
We say, well, anything that is more than

00:01:46.991 --> 00:01:49.440
one and a half times
the interquartile range

00:01:49.440 --> 00:01:52.483
from below Q-one or above Q-three,

00:01:52.483 --> 00:01:54.602
well, those are going to be outliers.

00:01:54.602 --> 00:01:56.325
Well, what am I talking about?

00:01:56.325 --> 00:01:58.021
Well, let's actually, let's
figure out the median,

00:01:58.021 --> 00:01:59.686
Q-one and Q-three here.

00:01:59.686 --> 00:02:01.715
Then we can figure out
the interquartile range.

00:02:01.715 --> 00:02:03.741
And then we can figure
out by that definition,

00:02:03.741 --> 00:02:05.950
what is going to be an outlier?

00:02:05.950 --> 00:02:07.563
And if that all made sense to you so far,

00:02:07.563 --> 00:02:08.863
I encourage you to pause this video

00:02:08.863 --> 00:02:10.372
and try to work through it on your own,

00:02:10.372 --> 00:02:13.176
or I'll do it for you right now.

00:02:13.176 --> 00:02:16.350
All right, so what's the median here?

00:02:16.350 --> 00:02:17.551
Well, the median is the middle number.

00:02:17.551 --> 00:02:20.626
We have 15 numbers, so the
middle number is going to be

00:02:20.626 --> 00:02:22.698
whatever number has seven on either side.

00:02:22.698 --> 00:02:24.307
So it's gonna be the eighth number.

00:02:24.307 --> 00:02:27.980
One, two, three, four, five, six, seven.

00:02:27.980 --> 00:02:29.013
Is that right?

00:02:29.013 --> 00:02:32.900
Yep, six, seven, so that's the median.

00:02:32.900 --> 00:02:35.619
And then you have one, two,
three, four, five, six, seven

00:02:35.619 --> 00:02:36.836
numbers on the right side too.

00:02:36.836 --> 00:02:41.064
So that is the median,
sometimes called Q-two.

00:02:41.064 --> 00:02:43.297
That is our median.

00:02:43.297 --> 00:02:44.928
Now what is Q-one?

00:02:44.928 --> 00:02:47.927
Well, Q-one is going to be the
middle of this first group.

00:02:47.927 --> 00:02:50.347
This first group has seven numbers in it.

00:02:50.347 --> 00:02:53.163
And so the middle is going
to be the fourth number.

00:02:53.163 --> 00:02:54.850
It has three and three,

00:02:54.850 --> 00:02:56.649
three to the left, three to the right.

00:02:56.649 --> 00:02:58.066
So that is Q-one.

00:02:59.507 --> 00:03:00.710
And then Q-three is going

00:03:00.710 --> 00:03:02.674
to be the middle of this upper group.

00:03:02.674 --> 00:03:04.178
Well, that also has seven numbers in it.

00:03:04.178 --> 00:03:06.267
So the middle is going
to be right over there.

00:03:06.267 --> 00:03:08.340
It has three on either side.

00:03:08.340 --> 00:03:09.923
So that is Q-three.

00:03:11.670 --> 00:03:14.239
Now what is the interquartile
range going to be?

00:03:14.239 --> 00:03:15.822
Interquartile range

00:03:16.911 --> 00:03:19.377
is going to be equal to

00:03:19.377 --> 00:03:20.661
Q-three

00:03:20.661 --> 00:03:21.661
minus Q-one,

00:03:22.631 --> 00:03:24.610
the difference between 18 and 13.

00:03:24.610 --> 00:03:26.110
Between 18 and 13,

00:03:26.991 --> 00:03:30.055
well, that is going to be 18 minus 13,

00:03:30.055 --> 00:03:32.085
which is equal to five.

00:03:32.085 --> 00:03:33.996
Now to figure out outliers,

00:03:33.996 --> 00:03:36.415
well, outliers are gonna
be anything that is below.

00:03:36.415 --> 00:03:37.415
So outliers,

00:03:38.579 --> 00:03:39.996
outliers,

00:03:39.996 --> 00:03:42.453
are going to be less than

00:03:42.453 --> 00:03:43.286
our Q-one

00:03:44.554 --> 00:03:45.387
minus 1.5,

00:03:46.620 --> 00:03:49.120
times our interquartile range.

00:03:50.593 --> 00:03:53.048
And this, once again, this
isn't some rule of the universe.

00:03:53.048 --> 00:03:54.246
This is something that statisticians

00:03:54.246 --> 00:03:55.346
have kind of said, well,

00:03:55.346 --> 00:03:57.489
if we want to have a better
definition for outliers,

00:03:57.489 --> 00:03:59.143
let's just agree that
it's something that's

00:03:59.143 --> 00:04:00.408
more than one and half times

00:04:00.408 --> 00:04:02.662
the interquartile range below Q-one.

00:04:02.662 --> 00:04:03.926
Or,

00:04:03.926 --> 00:04:07.751
or an outlier could be
greater than Q-three

00:04:07.751 --> 00:04:11.629
plus one and half times
the interquartile range,

00:04:11.629 --> 00:04:13.730
interquartile range.

00:04:13.730 --> 00:04:15.102
And once again, this is somewhat,

00:04:15.102 --> 00:04:16.929
you know, people just
decided it felt right.

00:04:16.929 --> 00:04:18.488
One could argue it should be 1.6.

00:04:18.488 --> 00:04:22.353
Or one could argue it should
be one, or two, or whatever.

00:04:22.353 --> 00:04:25.040
But this is what people
have tended to agree on.

00:04:25.040 --> 00:04:26.888
So let's think about
what these numbers are.

00:04:26.888 --> 00:04:27.973
Q-one we already know.

00:04:27.973 --> 00:04:30.058
So this is going to be 13

00:04:30.058 --> 00:04:33.927
minus 1.5 times our interquartile range.

00:04:33.927 --> 00:04:36.517
Our interquartile range here is five.

00:04:36.517 --> 00:04:39.600
So it's 1.5 times five, which is 7.5.

00:04:43.020 --> 00:04:44.270
So this is 7.5.

00:04:45.966 --> 00:04:47.716
13 minus 7.5 is what?

00:04:48.891 --> 00:04:50.592
13 minus seven is six,

00:04:50.592 --> 00:04:53.566
and then you subtract another .5, is 5.5.

00:04:53.566 --> 00:04:55.766
So we have outliers,

00:04:55.766 --> 00:04:57.262
outliers.

00:04:57.262 --> 00:04:58.095
Outliers

00:04:59.135 --> 00:05:01.052
would be less than 5.5.

00:05:02.617 --> 00:05:03.868
Or

00:05:03.868 --> 00:05:06.028
the Q-three is 18,

00:05:06.028 --> 00:05:08.111
this is, once again, 7.5.

00:05:09.726 --> 00:05:10.643
18 plus 7.5

00:05:12.118 --> 00:05:12.951
is 25.5,

00:05:14.287 --> 00:05:15.287
or outliers,

00:05:16.855 --> 00:05:18.938
outliers greater than 25,

00:05:20.698 --> 00:05:21.531
25.5.

00:05:22.507 --> 00:05:24.331
So based on this, we have a,

00:05:24.331 --> 00:05:26.436
kind of a numerical definition
for what's an outlier.

00:05:26.436 --> 00:05:28.373
We're not just subjectively saying,

00:05:28.373 --> 00:05:29.854
well, this feels right
or that feels right.

00:05:29.854 --> 00:05:32.595
And based on this, we
only have two outliers,

00:05:32.595 --> 00:05:36.795
that only these two
ones are less than 5.5.

00:05:36.795 --> 00:05:40.008
Only these two ones are less than 5.5.

00:05:40.008 --> 00:05:42.595
This is the cutoff, right over here.

00:05:42.595 --> 00:05:45.070
So this dot just happened to make it.

00:05:45.070 --> 00:05:48.013
And we don't have any
outliers on the high side.

00:05:48.013 --> 00:05:49.722
Now another thing to think about

00:05:49.722 --> 00:05:51.998
is drawing box-and-whiskers plots

00:05:51.998 --> 00:05:54.353
based on Q-one, our median, our range,

00:05:54.353 --> 00:05:55.567
all the range of numbers.

00:05:55.567 --> 00:05:56.650
And you could do it either

00:05:56.650 --> 00:05:58.351
taking in consideration your outliers

00:05:58.351 --> 00:06:02.370
or not taking into
consideration your outliers.

00:06:02.370 --> 00:06:05.267
So there's a couple of
ways that we can do it.

00:06:05.267 --> 00:06:08.909
So let me actually clear,
let me clear all of this.

00:06:08.909 --> 00:06:11.645
We've figured out all of this stuff.

00:06:11.645 --> 00:06:14.187
So let me clear all of that out.

00:06:14.187 --> 00:06:17.587
And let's actually draw
a box-and-whiskers plot.

00:06:17.587 --> 00:06:19.254
So I'll put another,

00:06:22.335 --> 00:06:24.305
another, actually let me do two here.

00:06:24.305 --> 00:06:25.222
That's one,

00:06:26.259 --> 00:06:28.852
and then let me put
another one down there.

00:06:28.852 --> 00:06:30.779
And then this is another.

00:06:30.779 --> 00:06:32.077
Now if we were to just draw

00:06:32.077 --> 00:06:35.307
a classic box-and-whiskers plot here,

00:06:35.307 --> 00:06:37.745
we would say, all right,
our median's at 14.

00:06:37.745 --> 00:06:39.207
And actually, I'll do it both ways.

00:06:39.207 --> 00:06:40.468
Our median's at 14.

00:06:40.468 --> 00:06:42.073
Median's at 14.

00:06:42.073 --> 00:06:43.631
Q-one's at 13.

00:06:43.631 --> 00:06:45.224
Q-one's at 13,

00:06:45.224 --> 00:06:46.905
and Q-one's at 13.

00:06:46.905 --> 00:06:48.453
Q-three is at 18.

00:06:48.453 --> 00:06:50.015
Q-three is at 18,

00:06:50.015 --> 00:06:51.042
Q-three is 18.

00:06:51.042 --> 00:06:52.729
So that's the box part.

00:06:52.729 --> 00:06:55.151
Now let me draw that as an actual,

00:06:55.151 --> 00:06:57.886
let me actually draw that as a box.

00:06:57.886 --> 00:06:59.469
So my best attempt,

00:07:00.655 --> 00:07:01.651
there you go.

00:07:01.651 --> 00:07:03.146
That's the box.

00:07:03.146 --> 00:07:05.753
And this is also a box.

00:07:05.753 --> 00:07:08.049
So far, I'm doing the exact same thing.

00:07:08.049 --> 00:07:09.607
Now if we don't want to consider outliers,

00:07:09.607 --> 00:07:11.373
we would say, well, what's
the entire range here?

00:07:11.373 --> 00:07:14.430
Well, we have things that go
from one all the way to 19.

00:07:14.430 --> 00:07:15.955
So one way to do it is to, hey,

00:07:15.955 --> 00:07:17.661
we start at one.

00:07:17.661 --> 00:07:19.940
And so our entire range, we go,

00:07:19.940 --> 00:07:22.471
actually let me draw it a
little bit better than that.

00:07:22.471 --> 00:07:24.691
We're going all the way,

00:07:24.691 --> 00:07:26.358
all the way from one

00:07:27.974 --> 00:07:28.807
to 19.

00:07:30.128 --> 00:07:32.256
Now in this one, we're
including everything.

00:07:32.256 --> 00:07:35.046
We're including even these two outliers.

00:07:35.046 --> 00:07:37.003
But if we don't want to
include those outliers,

00:07:37.003 --> 00:07:38.918
we want to make it clear
that they're outliers,

00:07:38.918 --> 00:07:40.454
well, let's not include them.

00:07:40.454 --> 00:07:42.671
And what we can do instead is say,

00:07:42.671 --> 00:07:45.978
all right, including
(chuckles) our non-outliers,

00:07:45.978 --> 00:07:47.843
we would start at six

00:07:47.843 --> 00:07:50.268
'cause six we're saying
is in our data set,

00:07:50.268 --> 00:07:52.069
but it is not an outlier.

00:07:52.069 --> 00:07:54.156
Let me make this look better.

00:07:54.156 --> 00:07:55.893
So we're gonna,

00:07:55.893 --> 00:07:57.143
we are going to

00:07:59.285 --> 00:08:02.452
start at six and go all the way to 19.

00:08:04.053 --> 00:08:05.689
And then to say that
we have these outliers,

00:08:05.689 --> 00:08:09.453
we would put this, we
have outliers over there.

00:08:09.453 --> 00:08:11.586
So once again, this is
a box-and-whiskers plot

00:08:11.586 --> 00:08:14.167
of the same data set without outliers.

00:08:14.167 --> 00:08:15.875
And this is one where we make specific,

00:08:15.875 --> 00:08:19.875
we make it clear where
the outliers actually are.


Techniques for generating a simple random sample

WEBVTT
Kind: captions
Language: en

00:00:00.188 --> 00:00:01.876
- [Teacher] Let's say that your school

00:00:01.876 --> 00:00:05.560
has a population of 80 students in it.

00:00:05.560 --> 00:00:06.460
Maybe it's not your whole school.

00:00:06.460 --> 00:00:08.178
Maybe it's just your grade.

00:00:08.178 --> 00:00:12.110
So there's 80 students in your population

00:00:12.110 --> 00:00:15.018
and you wanna get an estimate
of the average height

00:00:15.018 --> 00:00:17.595
in your population and
you think it's too hard

00:00:17.595 --> 00:00:20.397
for you to go and measure
the height of all 80 students

00:00:20.397 --> 00:00:22.485
so you decide to find a simple

00:00:22.485 --> 00:00:24.819
or take a simple random sample.

00:00:24.819 --> 00:00:27.521
You think it's reasonable
for you to measure

00:00:27.521 --> 00:00:29.575
the heights of 30 of these students

00:00:29.575 --> 00:00:33.477
and so what you wanna
do is randomly sample

00:00:33.477 --> 00:00:37.658
30 of the 80 students and
take their average height

00:00:37.658 --> 00:00:41.085
and say, well, that's probably
a pretty good estimate

00:00:41.085 --> 00:00:42.805
for the population parameter

00:00:42.805 --> 00:00:46.076
for the average height
of the entire population.

00:00:46.076 --> 00:00:47.695
So once you decide to do this,

00:00:47.695 --> 00:00:49.857
you say, well, how do I
select those 30 students

00:00:49.857 --> 00:00:52.474
and how do I select it so that I feel good

00:00:52.474 --> 00:00:54.904
that it is actually random?

00:00:54.904 --> 00:00:58.564
And there's several ways
that you could approach this.

00:00:58.564 --> 00:01:00.981
One way to do it is associate

00:01:02.395 --> 00:01:04.770
every person in your school

00:01:04.770 --> 00:01:07.469
with a piece of paper
and put 'em all in a bowl

00:01:07.469 --> 00:01:09.593
and then pick 'em out so let's do that.

00:01:09.593 --> 00:01:12.303
So let's say this is alphabetically

00:01:12.303 --> 00:01:13.327
the first person in the school.

00:01:13.327 --> 00:01:14.677
They're on a slip of paper

00:01:14.677 --> 00:01:16.887
then the next slip of
paper gets the next person

00:01:16.887 --> 00:01:18.360
and you're gonna go all the way down

00:01:18.360 --> 00:01:20.856
so you're gonna have 80 pieces of paper.

00:01:20.856 --> 00:01:22.575
They all should be the same size

00:01:22.575 --> 00:01:24.581
and then you throw them all,

00:01:24.581 --> 00:01:28.139
you throw them all into
a bowl of some kind

00:01:28.139 --> 00:01:30.408
and this seems like a
very basic way of doing it

00:01:30.408 --> 00:01:32.171
but it's actually a pretty effective way

00:01:32.171 --> 00:01:34.505
of getting a simple,

00:01:34.505 --> 00:01:36.505
of getting a simple random sample.

00:01:36.505 --> 00:01:38.101
So I'll try to draw a little,

00:01:38.101 --> 00:01:41.010
I don't know, that looks like
a fish bowl or something.

00:01:41.010 --> 00:01:43.179
Alright, so that's our bowl.

00:01:43.179 --> 00:01:45.882
And so all the pieces of paper go in there

00:01:45.882 --> 00:01:49.237
and then you put a blindfold on someone

00:01:49.237 --> 00:01:51.746
and they can't feel what names are there

00:01:51.746 --> 00:01:54.218
and so they should pick out the first 30

00:01:54.218 --> 00:01:55.810
without replacing them

00:01:55.810 --> 00:01:57.934
because you obviously
don't wanna pick the same,

00:01:57.934 --> 00:02:00.556
you don't wanna pick
out the same name twice.

00:02:00.556 --> 00:02:02.232
And those 30 names that you pick,

00:02:02.232 --> 00:02:04.360
that would be your simple random sample

00:02:04.360 --> 00:02:06.126
and then you could measure their heights

00:02:06.126 --> 00:02:08.999
to estimate the average
height for the population.

00:02:08.999 --> 00:02:12.565
This would be a completely
legitimate way of doing it.

00:02:12.565 --> 00:02:14.487
Other ways that you could do it,

00:02:14.487 --> 00:02:15.879
if you have a computer or a calculator,

00:02:15.879 --> 00:02:17.719
you could use a random number generator

00:02:17.719 --> 00:02:21.495
and the random functions on
computer programming languages

00:02:21.495 --> 00:02:23.964
on your calculator, they
tend to be something,

00:02:23.964 --> 00:02:27.402
someplace you'll see
something like a math.rand

00:02:27.402 --> 00:02:29.165
Rand's short for random.

00:02:29.165 --> 00:02:32.287
You might see something like random.

00:02:32.287 --> 00:02:33.962
You might see,

00:02:33.962 --> 00:02:37.534
you might see something like random

00:02:37.534 --> 00:02:39.456
without anything passed into it.

00:02:39.456 --> 00:02:43.271
It might give you a number
between zero and one

00:02:43.271 --> 00:02:45.770
or zero and 100 and you
have to be very careful

00:02:45.770 --> 00:02:47.325
on how you use this to
make sure that you have

00:02:47.325 --> 00:02:50.742
an even chance of picking certain numbers

00:02:51.947 --> 00:02:54.121
but what you would do in this situation

00:02:54.121 --> 00:02:57.031
if you had access to some
random number generator

00:02:57.031 --> 00:02:58.700
and it could even pick out a random number

00:02:58.700 --> 00:03:01.654
between one and 80 including one and 80

00:03:01.654 --> 00:03:03.970
is you would maybe line up

00:03:03.970 --> 00:03:06.078
all the students' names alphabetically

00:03:06.078 --> 00:03:08.427
and so the first student alphabetically

00:03:08.427 --> 00:03:10.260
assigned the number 01

00:03:12.609 --> 00:03:14.085
and you could just say one

00:03:14.085 --> 00:03:15.558
if you're using a random number generator

00:03:15.558 --> 00:03:17.465
but I'll use two digits for it

00:03:17.465 --> 00:03:19.254
just because it'll be
useful and consistent

00:03:19.254 --> 00:03:21.024
and in a little bit, we'll
use another technique

00:03:21.024 --> 00:03:22.527
where it's gonna be nice to be consistent

00:03:22.527 --> 00:03:23.978
with our number of digits.

00:03:23.978 --> 00:03:26.634
And so the next one, 02

00:03:26.634 --> 00:03:28.967
and you go all the way to 79

00:03:30.440 --> 00:03:32.573
and all the way to 80

00:03:32.573 --> 00:03:34.250
and then you use your
random number generator

00:03:34.250 --> 00:03:37.694
to keep generating numbers from one to 80

00:03:37.694 --> 00:03:40.232
and as long as you don't get repeats,

00:03:40.232 --> 00:03:44.033
you pick the first 30 to be
your actual random sample.

00:03:44.033 --> 00:03:45.505
Another related technique

00:03:45.505 --> 00:03:47.017
which is a little bit more old school

00:03:47.017 --> 00:03:48.982
but is definitely the way that
it has been done in the past

00:03:48.982 --> 00:03:50.803
and even done now sometimes

00:03:50.803 --> 00:03:52.643
is to use a random digit table.

00:03:52.643 --> 00:03:55.304
You still start with
these number associations

00:03:55.304 --> 00:03:57.310
with each student in the class

00:03:57.310 --> 00:04:01.965
and then you use a randomly
generated list of numbers

00:04:01.965 --> 00:04:03.357
and so let's say that's our

00:04:03.357 --> 00:04:04.954
randomly generated list of numbers

00:04:04.954 --> 00:04:07.371
and it keeps going well beyond this

00:04:07.371 --> 00:04:08.518
and you start at the beginning

00:04:08.518 --> 00:04:10.520
and you say, okay, we're
interested in getting,

00:04:10.520 --> 00:04:14.363
we're interested in getting
30 two digit numbers

00:04:14.363 --> 00:04:16.942
from one to 80 including one and 80.

00:04:16.942 --> 00:04:18.709
So one technique that you could use

00:04:18.709 --> 00:04:20.945
is you start right at the
beginning and you could say,

00:04:20.945 --> 00:04:23.814
alright, this is a randomly
generated list of numbers.

00:04:23.814 --> 00:04:26.393
So the first number here is 59.

00:04:26.393 --> 00:04:28.486
Is 59 between one and 80?

00:04:28.486 --> 00:04:29.766
Sure is, as long as we,

00:04:29.766 --> 00:04:31.391
if this was a 01, that would have worked.

00:04:31.391 --> 00:04:33.263
If this was an 80, that would have worked.

00:04:33.263 --> 00:04:35.034
If this was a 00, it wouldn't have worked.

00:04:35.034 --> 00:04:37.117
If this was an 81, it
wouldn't have worked.

00:04:37.117 --> 00:04:39.446
This would be our, this right over here,

00:04:39.446 --> 00:04:41.801
that would be our first name that we,

00:04:41.801 --> 00:04:43.396
you could imagine the same

00:04:43.396 --> 00:04:45.034
as picking that first name out of the hat,

00:04:45.034 --> 00:04:48.164
whoever's associated with number 59.

00:04:48.164 --> 00:04:49.418
Now, you would move on.

00:04:49.418 --> 00:04:51.177
You get the next two digits.

00:04:51.177 --> 00:04:53.224
The next two digits are 83.

00:04:53.224 --> 00:04:55.801
They don't follow into
our range from one to 80

00:04:55.801 --> 00:04:57.949
so we're not going to use it

00:04:57.949 --> 00:05:00.321
then you would look at
the next two digits.

00:05:00.321 --> 00:05:02.654
So we get a five and a nine.

00:05:03.799 --> 00:05:06.687
Well, that fits in our range
but we already picked 59.

00:05:06.687 --> 00:05:07.929
We already picked person 59

00:05:07.929 --> 00:05:11.730
so we're not gonna pick 59
again so we keep moving on

00:05:11.730 --> 00:05:13.147
then we get a 37.

00:05:14.268 --> 00:05:16.590
Well, that's in our range,
we haven't picked that yet.

00:05:16.590 --> 00:05:18.923
We do that then we get a 00.

00:05:21.341 --> 00:05:23.605
Once again, not in our range.

00:05:23.605 --> 00:05:24.915
I think you see where this is going.

00:05:24.915 --> 00:05:26.381
91.

00:05:26.381 --> 00:05:28.382
Not in our range.

00:05:28.382 --> 00:05:29.215
23.

00:05:30.307 --> 00:05:32.601
It's in our range and
we haven't picked it yet

00:05:32.601 --> 00:05:35.011
so we're gonna pick the 23.

00:05:35.011 --> 00:05:37.134
I think you see where this is going.

00:05:37.134 --> 00:05:38.879
We're gonna keep going down this list

00:05:38.879 --> 00:05:42.526
in the way that I've just
described until we get,

00:05:42.526 --> 00:05:44.617
until we get 30 of these.

00:05:44.617 --> 00:05:45.450
We've just gotten three.

00:05:45.450 --> 00:05:47.482
We just have to keep on going

00:05:47.482 --> 00:05:49.296
and this isn't an exhaustive list

00:05:49.296 --> 00:05:52.313
of all of the different ways
that you can get random numbers

00:05:52.313 --> 00:05:55.515
but it starts to give you some
techniques in your toolkit

00:05:55.515 --> 00:05:57.440
and you might say, oh, well,
why don't I just randomly

00:05:57.440 --> 00:05:59.197
come up with some numbers in my head?

00:05:59.197 --> 00:06:01.979
And I would really suggest
that you don't do that

00:06:01.979 --> 00:06:05.825
because humans are famously
bad at being truly random

00:06:05.825 --> 00:06:08.236
and you might wanna do something like

00:06:08.236 --> 00:06:11.440
even use something that you
think is a random process

00:06:11.440 --> 00:06:13.402
but you realize later
that it wasn't as random

00:06:13.402 --> 00:06:14.343
as you thought.

00:06:14.343 --> 00:06:16.762
So once again, multiple techniques

00:06:16.762 --> 00:06:20.160
but these are some of the
I would say best practices

00:06:20.160 --> 00:06:24.077
for actually generating
a simple random sample.


Examples of bias in surveys

WEBVTT
Kind: captions
Language: en

00:00:00.290 --> 00:00:01.790
- [Instructor] We're told
that David hosts a podcast

00:00:01.790 --> 00:00:04.580
and he is curious how much
his listeners like his show.

00:00:04.580 --> 00:00:06.470
He decides to start with an online poll.

00:00:06.470 --> 00:00:08.873
He asks his listeners to visit his website

00:00:08.873 --> 00:00:10.393
and participate in the poll.

00:00:10.393 --> 00:00:14.511
The poll shows that 89% of
about 200 respondents "love"

00:00:14.511 --> 00:00:15.534
his show.

00:00:15.534 --> 00:00:19.725
What is the most concerning
source of bias in this scenario?

00:00:19.725 --> 00:00:21.723
And well, like always,
pause this video and see

00:00:21.723 --> 00:00:23.358
if you can figure it out on your own

00:00:23.358 --> 00:00:25.109
and then we'll work through it together.

00:00:25.109 --> 00:00:26.295
Let's think about what's going on.

00:00:26.295 --> 00:00:28.845
He has this population
of listeners, right?

00:00:28.845 --> 00:00:31.409
I'll assume that the number
of listeners is more than 200.

00:00:31.409 --> 00:00:33.956
And he says, "Hey I want to find a sample,

00:00:33.956 --> 00:00:36.679
"and I can't ask all of my listeners."

00:00:36.679 --> 00:00:38.076
Who knows, maybe he has 10,000 listeners,

00:00:38.076 --> 00:00:39.983
they don't tell us that, but let's say

00:00:39.983 --> 00:00:41.244
there's 10,000 listeners here.

00:00:41.244 --> 00:00:42.427
And he says, "Well I
want to get an indication

00:00:42.427 --> 00:00:44.196
"of what percentage like my show.

00:00:44.196 --> 00:00:46.031
"So I need a sample."

00:00:46.031 --> 00:00:48.115
But instead of taking
a truly random sample,

00:00:48.115 --> 00:00:50.235
he asks them to volunteer.

00:00:50.235 --> 00:00:52.593
He asks his listeners
to visit his website.

00:00:52.593 --> 00:00:56.073
So that's classic volunteer
response sampling.

00:00:56.073 --> 00:00:58.610
This is not random because who decides

00:00:58.610 --> 00:01:00.556
to go to his website and
listen to what he just said,

00:01:00.556 --> 00:01:02.257
and maybe even has access to a computer.

00:01:02.257 --> 00:01:03.663
That's not random.

00:01:03.663 --> 00:01:05.219
In fact, the people
more likely to do that,

00:01:05.219 --> 00:01:08.965
so these are the people out of the 10,000,

00:01:08.965 --> 00:01:12.929
these are the 200 responders
here who decided to do it.

00:01:12.929 --> 00:01:16.272
These are more likely to be
the people who already like

00:01:16.272 --> 00:01:18.493
David or like to listen to
what he tells them to do.

00:01:18.493 --> 00:01:21.312
The people, the listeners
who are not into David

00:01:21.312 --> 00:01:25.062
or don't want to do what
he tells them to do,

00:01:26.285 --> 00:01:29.057
they're unlikely to say, "Oh,
I'm not really into David

00:01:29.057 --> 00:01:30.781
"and I don't like him
telling me what to do,

00:01:30.781 --> 00:01:32.050
"but hey, I'm gonna go
to his website anyway,

00:01:32.050 --> 00:01:33.186
"I'm gonna fill out that poll."

00:01:33.186 --> 00:01:34.545
That's less likely.

00:01:34.545 --> 00:01:36.351
Or you might get extremes,
people who really don't like him,

00:01:36.351 --> 00:01:37.927
might say, "I'm gonna
definitely go there."

00:01:37.927 --> 00:01:40.354
But in this case, I would
say that it's more likely

00:01:40.354 --> 00:01:42.641
your fans are gonna do
what you ask them to do

00:01:42.641 --> 00:01:45.911
and go to your website and
spend time on your website.

00:01:45.911 --> 00:01:50.078
And because of that, that 89%
is probably an overestimate.

00:01:51.132 --> 00:01:54.976
89% is probably an overestimate
of the number of listeners

00:01:54.976 --> 00:01:57.059
who really love his show.

00:01:57.955 --> 00:01:59.955
Cause you're more likely to
get the ones who love him

00:01:59.955 --> 00:02:02.182
to show up and fill
out that actual survey.

00:02:02.182 --> 00:02:04.115
Now these other forms of bias.

00:02:04.115 --> 00:02:06.853
Response bias, this is when
you're asking something

00:02:06.853 --> 00:02:09.340
that people don't necessarily
want to answer truthfully,

00:02:09.340 --> 00:02:12.752
or the way that it's phrased,
it might make someone

00:02:12.752 --> 00:02:15.833
respond, you see, in a biased way.

00:02:15.833 --> 00:02:17.874
Classic examples of this are like,

00:02:17.874 --> 00:02:22.195
"Have you lied to your
parents in the past week?"

00:02:22.195 --> 00:02:25.695
Or "have you ever cheated on your spouse."

00:02:27.193 --> 00:02:28.858
Something, "do you smoke?"

00:02:28.858 --> 00:02:30.943
Any of these things that
people might not want

00:02:30.943 --> 00:02:33.313
to answer completely truthfully
or they might be hiding

00:02:33.313 --> 00:02:34.965
from the world, they might
not just want to answer

00:02:34.965 --> 00:02:36.417
that truthfully on a survey.

00:02:36.417 --> 00:02:38.372
And so you're going to have response bias.

00:02:38.372 --> 00:02:40.535
But that's not the case right over here.

00:02:40.535 --> 00:02:43.800
And undercoverage is when
the way that you're sampling,

00:02:43.800 --> 00:02:47.181
you're definitely missing out
on an important constituency.

00:02:47.181 --> 00:02:49.590
Voluntary response
we're likely missing out

00:02:49.590 --> 00:02:51.903
on some important
constituencies, on some people

00:02:51.903 --> 00:02:53.473
who might not be into
going to your website,

00:02:53.473 --> 00:02:56.016
but undercoverage is where
it's a little bit more clear

00:02:56.016 --> 00:02:57.631
that that is happening.

00:02:57.631 --> 00:02:59.830
Let's do another case,
let's do another case,

00:02:59.830 --> 00:03:02.124
maybe an alternate reality
where David's trying

00:03:02.124 --> 00:03:04.370
to figure this out again,
he's still hosting a podcast,

00:03:04.370 --> 00:03:07.124
he's still curious how much
his listeners like his show,

00:03:07.124 --> 00:03:09.266
but he tries to take a different sample.

00:03:09.266 --> 00:03:12.698
He decides in this case, to
poll the next 100 listeners

00:03:12.698 --> 00:03:15.198
who send him fan emails.

00:03:15.198 --> 00:03:17.889
They don't all respond, but 94 out

00:03:17.889 --> 00:03:21.037
of the 97 listeners polled
said they "loved his show."

00:03:21.037 --> 00:03:24.041
What is the most concerning
source of bias in this scenario?

00:03:24.041 --> 00:03:27.193
Well this is a classic,
"Hey I have a group,

00:03:27.193 --> 00:03:28.971
"I have a sample sitting in front of me,

00:03:28.971 --> 00:03:30.805
"it's in my inbox in my email,
let me just go to them."

00:03:30.805 --> 00:03:32.624
Isn't that convenient?

00:03:32.624 --> 00:03:35.222
So this is classic convenience sample.

00:03:35.222 --> 00:03:36.689
And this isn't just like,

00:03:36.689 --> 00:03:39.136
hey, these are the first 100
people to walk through the door

00:03:39.136 --> 00:03:41.629
and there's, a lot of times you can argue

00:03:41.629 --> 00:03:43.078
why that might be not so random,

00:03:43.078 --> 00:03:45.864
but these're the next 100
listeners who sent him fan emails.

00:03:45.864 --> 00:03:47.701
(laughing)

00:03:47.701 --> 00:03:49.851
So this is convenience
sampling and the sample

00:03:49.851 --> 00:03:52.381
that you happen to use out of convenience

00:03:52.381 --> 00:03:55.399
is one that's going to be
very skewed to liking you.

00:03:55.399 --> 00:03:57.861
So once again, this is overestimating,

00:03:57.861 --> 00:04:00.368
overestimating the percent,

00:04:00.368 --> 00:04:02.951
the percent that love his show.

00:04:04.778 --> 00:04:07.318
Now nonresponse is when you
ask a certain number of people

00:04:07.318 --> 00:04:10.767
to fill out a survey or
to answer a questionnaire,

00:04:10.767 --> 00:04:14.379
and for some reason, some
percent do not fill it out.

00:04:14.379 --> 00:04:15.688
And you're like, "Wow,
who are those people?

00:04:15.688 --> 00:04:17.017
"Maybe they would have
said something important

00:04:17.017 --> 00:04:18.425
"and maybe their viewpoint

00:04:18.425 --> 00:04:20.361
"is not properly represented
in the overall number

00:04:20.361 --> 00:04:22.616
"that actually did fill it out."

00:04:22.616 --> 00:04:24.845
And there is some
nonresponse going on here.

00:04:24.845 --> 00:04:28.411
He asks 100 people who
sent fan emails to fill out

00:04:28.411 --> 00:04:31.338
the survey to say whether
they love it or not,

00:04:31.338 --> 00:04:33.318
97 fill it out.

00:04:33.318 --> 00:04:36.195
So there are three people who
did not fill out the survey.

00:04:36.195 --> 00:04:38.092
So there is some nonresponse going on

00:04:38.092 --> 00:04:39.984
that would be a source of bias,

00:04:39.984 --> 00:04:41.581
but it's not the most concerning.

00:04:41.581 --> 00:04:43.305
Right over here they're
asking us, fill out

00:04:43.305 --> 00:04:46.823
the most concerning source of bias,

00:04:46.823 --> 00:04:48.704
and the convenience sampling is definitely

00:04:48.704 --> 00:04:49.863
the biggest deal here.

00:04:49.863 --> 00:04:51.311
There were three people
who didn't respond,

00:04:51.311 --> 00:04:52.697
but that's not as big of a deal.

00:04:52.697 --> 00:04:54.930
Voluntary response sampling.

00:04:54.930 --> 00:04:59.855
Well, he didn't ask people,
like in the last example,

00:04:59.855 --> 00:05:03.355
"Hey, if you can go here and fill it out?"

00:05:05.335 --> 00:05:07.332
I guess, I take that back,
there is a little bit

00:05:07.332 --> 00:05:08.897
of voluntary response here,

00:05:08.897 --> 00:05:11.468
where he goes to these 100 people

00:05:11.468 --> 00:05:13.909
and he asks them to respond.

00:05:13.909 --> 00:05:18.052
And so you have the 97
people who chose to respond.

00:05:18.052 --> 00:05:21.986
But once again, that
could be a source of bias,

00:05:21.986 --> 00:05:23.907
but most of the 97 of
the 100 are responding,

00:05:23.907 --> 00:05:25.883
and once again, the most concerning thing

00:05:25.883 --> 00:05:28.189
is the convenience sampling,
which will once again,

00:05:28.189 --> 00:05:31.539
based on this sample that
he's happening to use

00:05:31.539 --> 00:05:33.794
out of convenience, is going to be a very,

00:05:33.794 --> 00:05:36.531
a significant overestimate in terms of

00:05:36.531 --> 00:05:40.698
representing the entire
population of his listeners.


Example of under coverage introducing bias

WEBVTT
Kind: captions
Language: en

00:00:00.288 --> 00:00:01.277
- [Instructor] A senator wanted to know

00:00:01.277 --> 00:00:02.740
about how people in her state

00:00:02.740 --> 00:00:05.108
felt about internet privacy issues.

00:00:05.108 --> 00:00:07.309
She conducted a poll by calling 100 people

00:00:07.309 --> 00:00:10.360
whose names were randomly
sampled from the phone book.

00:00:10.360 --> 00:00:12.729
Note that mobile phones
and unlisted numbers

00:00:12.729 --> 00:00:15.056
are not in phone books.

00:00:15.056 --> 00:00:17.410
The senator's office called those numbers

00:00:17.410 --> 00:00:21.395
until they got a response
from all 100 people chosen.

00:00:21.395 --> 00:00:23.679
The poll showed that 42% of respondents

00:00:23.679 --> 00:00:27.385
were very concerned
about internet privacy.

00:00:27.385 --> 00:00:29.378
What is the most concerning

00:00:29.378 --> 00:00:31.244
source of bias in this scenario?

00:00:31.244 --> 00:00:32.457
And we should also think about,

00:00:32.457 --> 00:00:35.522
well what kind of bias
does that likely introduce?

00:00:35.522 --> 00:00:38.601
Is this likely to be an
overestimate or an underestimate

00:00:38.601 --> 00:00:40.000
of the number of respondents?

00:00:40.000 --> 00:00:41.616
And maybe there's no bias here.

00:00:41.616 --> 00:00:43.107
But our choices,

00:00:43.107 --> 00:00:45.810
and no bias is not one of
the choices you can imagine,

00:00:45.810 --> 00:00:48.498
it's gonna be one of these three.

00:00:48.498 --> 00:00:49.947
So, I encourage you to pause this video

00:00:49.947 --> 00:00:51.229
and think about what we just said.

00:00:51.229 --> 00:00:52.650
Where we're a senator,

00:00:52.650 --> 00:00:54.183
we're trying to figure out what percentage

00:00:54.183 --> 00:00:56.008
of our response, of our constituents,

00:00:56.008 --> 00:00:58.418
are very concerned about internet privacy.

00:00:58.418 --> 00:01:00.410
And we go to the phone
book, we sample 100 people.

00:01:00.410 --> 00:01:02.403
We keep calling them until they answer,

00:01:02.403 --> 00:01:05.273
and we get that 42% are very concerned.

00:01:05.273 --> 00:01:08.101
So what's the source of bias?

00:01:08.101 --> 00:01:11.124
Alright, now let's work
through this together.

00:01:11.124 --> 00:01:14.231
So nonresponse is, would've been the case,

00:01:14.231 --> 00:01:16.516
if we selected these 100 people,

00:01:16.516 --> 00:01:18.550
and let's say only 50
people answered the phone

00:01:18.550 --> 00:01:20.124
and we didn't keep calling them.

00:01:20.124 --> 00:01:23.440
Then we'd say, well 50 of
the people who we sampled

00:01:23.440 --> 00:01:25.725
to answer our survey didn't even respond.

00:01:25.725 --> 00:01:27.048
There was a nonresponse there,

00:01:27.048 --> 00:01:28.832
what was there about those 50 people?

00:01:28.832 --> 00:01:30.448
Maybe there was something that
would've skewed the survey

00:01:30.448 --> 00:01:32.259
or actually if we, had we gotten them,

00:01:32.259 --> 00:01:34.390
it would've gotten maybe get better data.

00:01:34.390 --> 00:01:36.508
But in this case, they tell us,

00:01:36.508 --> 00:01:38.751
the senator's office called those numbers

00:01:38.751 --> 00:01:41.273
until they got a response
from all 100 people chosen.

00:01:41.273 --> 00:01:42.945
So the 100 people that they chose,

00:01:42.945 --> 00:01:44.770
they made sure they got a response.

00:01:44.770 --> 00:01:48.197
So nonresponse is not
going to be an issue here.

00:01:48.197 --> 00:01:50.733
Alright, next choice, undercoverage.

00:01:50.733 --> 00:01:52.432
Well undercoverage is
where you're not able

00:01:52.432 --> 00:01:55.177
to sample from part of the population.

00:01:55.177 --> 00:01:57.949
And a part of the population
that actually might,

00:01:57.949 --> 00:01:59.858
because you didn't sample it,

00:01:59.858 --> 00:02:01.516
it might introduce bias.

00:02:01.516 --> 00:02:03.717
Now let's think about what
happened in this situation.

00:02:03.717 --> 00:02:05.877
We are a senator.

00:02:05.877 --> 00:02:08.621
We want to sample all of our constituents,

00:02:08.621 --> 00:02:11.031
but we choose, we instead we sample

00:02:11.031 --> 00:02:11.895
from the constituents

00:02:11.895 --> 00:02:14.305
who happen to be listed in the phone book.

00:02:14.305 --> 00:02:17.203
So these are the people who happen to be

00:02:17.203 --> 00:02:19.286
listed in the phone book.

00:02:21.202 --> 00:02:23.821
And so we're not sampling from people

00:02:23.821 --> 00:02:25.521
who are not in the phone book,

00:02:25.521 --> 00:02:28.224
who maybe have landlines
and they're unlisted.

00:02:28.224 --> 00:02:30.035
And we're not sampling from people

00:02:30.035 --> 00:02:33.281
who don't have landlines,
who only have mobile phones.

00:02:33.281 --> 00:02:35.301
And you might say, well
why is that important?

00:02:35.301 --> 00:02:36.137
Well think about it,

00:02:36.137 --> 00:02:39.871
people who decide not to
list in the phone book

00:02:39.871 --> 00:02:41.612
or people who don't even have a landline,

00:02:41.612 --> 00:02:43.409
some of those people might be
a little bit more concerned

00:02:43.409 --> 00:02:45.095
about privacy than everyone else.

00:02:45.095 --> 00:02:47.923
They explicitly chose not to be listed.

00:02:47.923 --> 00:02:51.239
So undercoverage is
definitely a very concerning

00:02:51.239 --> 00:02:53.301
source of bias over here.

00:02:53.301 --> 00:02:55.962
We are sampling from only a subset

00:02:55.962 --> 00:02:58.163
of our entire population we care about.

00:02:58.163 --> 00:02:59.905
In particular, we're missing out on people

00:02:59.905 --> 00:03:01.939
who might care about privacy.

00:03:01.939 --> 00:03:04.057
And so I would say
because of undercoverage,

00:03:04.057 --> 00:03:07.094
42% is likely to be an underestimate

00:03:07.094 --> 00:03:09.337
of the people concerned
about internet privacy.

00:03:09.337 --> 00:03:11.649
Probably a higher proportion
of the people out here

00:03:11.649 --> 00:03:13.182
care about privacy,
because they're unlisted

00:03:13.182 --> 00:03:15.091
or they don't even have a landline.

00:03:15.091 --> 00:03:18.295
So, undercoverage, it
probably introduced bias,

00:03:18.295 --> 00:03:22.295
and it implies that
42% is an underestimate

00:03:25.568 --> 00:03:28.173
of the percentage of the
senator's constituents

00:03:28.173 --> 00:03:30.458
who care about internet privacy.

00:03:30.458 --> 00:03:34.625
Now the last question,
volunteer response sampling.

00:03:36.894 --> 00:03:38.148
Well this would be the case where you,

00:03:38.148 --> 00:03:39.681
you know the senator, I don't know,

00:03:39.681 --> 00:03:42.425
put a billboard out or just told someone,

00:03:42.425 --> 00:03:44.613
told a bunch of people,
maybe on her website,

00:03:44.613 --> 00:03:48.263
hey vote for this or
give us your information

00:03:48.263 --> 00:03:51.551
on how much you care
about internet privacy.

00:03:51.551 --> 00:03:54.992
And that would've been,
the source of bias there,

00:03:54.992 --> 00:03:56.586
is well who shows up on that website?

00:03:56.586 --> 00:03:57.721
Once again, if you did,

00:03:57.721 --> 00:04:00.006
hey come to my website and fill it out,

00:04:00.006 --> 00:04:03.531
you're only getting
information from a subset

00:04:03.531 --> 00:04:05.899
of your population who are choosing,

00:04:05.899 --> 00:04:07.473
who are volunteering.

00:04:07.473 --> 00:04:10.162
That is not the situation
that she did over here.

00:04:10.162 --> 00:04:12.698
She didn't ask 100 people to volunteer.

00:04:12.698 --> 00:04:16.432
Her team went out and got
them from the phone book.

00:04:16.432 --> 00:04:19.873
So this was definitely
a case of undercoverage.