Contingency table chi-square test

WEBVTT
Kind: captions
Language: en

00:00:00.509 --> 00:00:02.050
Let's say there are
a couple of herbs

00:00:02.050 --> 00:00:06.500
that people believe
help prevent the flu.

00:00:06.500 --> 00:00:09.690
So, to test this, what we
do is we wait for flu season

00:00:09.690 --> 00:00:12.890
and we randomly assign people
to three different groups.

00:00:12.890 --> 00:00:15.480
And over the course
of flu season,

00:00:15.480 --> 00:00:18.150
we have them either in
one group taking herb one,

00:00:18.150 --> 00:00:20.780
in the second group taking herb
two, and in the third group

00:00:20.780 --> 00:00:21.691
they take a placebo.

00:00:21.691 --> 00:00:23.940
And if you don't know what
a placebo is it's something

00:00:23.940 --> 00:00:27.029
that, to the patient or to
the person participating,

00:00:27.029 --> 00:00:29.320
it feels like they're taking
something that you've told

00:00:29.320 --> 00:00:31.420
them might help them,
but it does nothing.

00:00:31.420 --> 00:00:34.050
It could be just a sugar pill,
just so it feels like medicine.

00:00:34.050 --> 00:00:36.591
The reason why you would even
go through the effort of giving

00:00:36.591 --> 00:00:38.530
them something is because
oftentimes there's

00:00:38.530 --> 00:00:40.946
something called a placebo
effect, where people get better

00:00:40.946 --> 00:00:43.899
just because they're being
told that they're being given

00:00:43.899 --> 00:00:45.440
something that will
make them better.

00:00:45.440 --> 00:00:51.570
So this could, right here,
just be a sugar pill,

00:00:51.570 --> 00:00:53.520
and a very small amount
of sugar so it really

00:00:53.520 --> 00:00:58.250
can't affect the actual
likelihood of getting the flu.

00:00:58.250 --> 00:01:01.270
So over here we have a
table, and this is actually

00:01:01.270 --> 00:01:02.590
called a contingency table.

00:01:08.770 --> 00:01:12.580
And it has on it in each group
the number that got sick,

00:01:12.580 --> 00:01:14.180
the number that didn't get sick.

00:01:14.180 --> 00:01:17.080
And so we also can from this
calculate the total number.

00:01:17.080 --> 00:01:22.150
So in group one, we had
a total of 120 people.

00:01:22.150 --> 00:01:26.840
In group two, we had a total
of 30 plus 110 is 140 people.

00:01:26.840 --> 00:01:29.630
And in the placebo group, the
group that just got the sugar

00:01:29.630 --> 00:01:34.250
pill, we had a
total of 120 people.

00:01:34.250 --> 00:01:36.000
And then we could also
tabulate the number

00:01:36.000 --> 00:01:38.780
of people, the total number
of people, that got sick.

00:01:38.780 --> 00:01:43.790
So that's 20 plus 30
is 50 plus 30 is 80.

00:01:43.790 --> 00:01:45.940
This is the total
column right over here.

00:01:45.940 --> 00:01:48.580
And then the total people
that didn't get sick over here

00:01:48.580 --> 00:01:56.130
is 100 plus 110 is
210 plus 90 is 300,

00:01:56.130 --> 00:01:59.220
and then the total
people here are 380.

00:01:59.220 --> 00:02:02.684
Both this column and this
row should add up to 380.

00:02:02.684 --> 00:02:04.350
So with that out of
the way, let's think

00:02:04.350 --> 00:02:07.140
about how we can use this
information in the contingency

00:02:07.140 --> 00:02:10.310
table and our knowledge of
the chi-square distribution

00:02:10.310 --> 00:02:12.880
to come up with some conclusion.

00:02:12.880 --> 00:02:14.960
So let's just make
a null hypothesis.

00:02:14.960 --> 00:02:17.140
Our null hypothesis is
that the herbs do nothing.

00:02:20.010 --> 00:02:22.940
Let's just assume-- let me
get some space here-- so

00:02:22.940 --> 00:02:29.240
let's assume the null hypothesis
that the herbs do nothing.

00:02:31.800 --> 00:02:33.880
And then we have our
alternative hypothesis,

00:02:33.880 --> 00:02:36.130
or alternate hypothesis,
that the herbs do something.

00:02:39.280 --> 00:02:41.664
Notice I don't even care
whether they actually improve.

00:02:41.664 --> 00:02:43.080
I'm just saying
they do something.

00:02:43.080 --> 00:02:45.370
They might even increase your
likelihood of getting the flu.

00:02:45.370 --> 00:02:47.000
We're not testing whether
they're actually good.

00:02:47.000 --> 00:02:48.960
We're just saying, are
they different than just

00:02:48.960 --> 00:02:50.750
doing nothing.

00:02:50.750 --> 00:02:53.650
So like we do with all
of our hypothesis tests,

00:02:53.650 --> 00:02:55.950
let's just assume the null.

00:02:55.950 --> 00:03:01.600
We're going to assume the null
and, given that assumption,

00:03:01.600 --> 00:03:05.590
figure out if the
likelihood of getting data

00:03:05.590 --> 00:03:08.140
like this or more
extreme is really low.

00:03:08.140 --> 00:03:10.080
And if it is really
low, then we will

00:03:10.080 --> 00:03:11.920
reject the null hypothesis.

00:03:11.920 --> 00:03:14.120
And in this test, like
every hypothesis test,

00:03:14.120 --> 00:03:15.720
we need a significance level.

00:03:15.720 --> 00:03:18.100
And let's say our significance
level we care about

00:03:18.100 --> 00:03:24.580
for whatever reason
is 10% or 0.10.

00:03:24.580 --> 00:03:27.890
That's the significance
level that we care about.

00:03:27.890 --> 00:03:30.605
Now to do this, we
have to calculate

00:03:30.605 --> 00:03:34.875
a chi-square statistic for
this contingency table.

00:03:34.875 --> 00:03:36.750
And to do that, we do
it very similar to what

00:03:36.750 --> 00:03:38.291
we did with the
restaurant situation.

00:03:38.291 --> 00:03:42.130
We figure out, assuming
the null hypothesis,

00:03:42.130 --> 00:03:43.570
the expected
results you would've

00:03:43.570 --> 00:03:46.440
gotten in each of these cells.

00:03:46.440 --> 00:03:49.262
You could view each of
these entries as a cell.

00:03:49.262 --> 00:03:50.720
You know that's
what we do with it.

00:03:50.720 --> 00:03:52.553
You call each of those
entries in Excel also

00:03:52.553 --> 00:03:54.640
a cell, each of the
entries in a table.

00:03:54.640 --> 00:03:58.420
What we do is we figure out
what the expected value would

00:03:58.420 --> 00:04:02.970
have been if you do assume
the null hypothesis.

00:04:02.970 --> 00:04:05.430
Then we find the
squared distance

00:04:05.430 --> 00:04:08.960
from that expected value, and
we, I guess you could call it,

00:04:08.960 --> 00:04:10.780
normalize it by
the expected value.

00:04:10.780 --> 00:04:13.320
Take the sum of all
of those differences,

00:04:13.320 --> 00:04:16.454
and if those squares
differences are really big,

00:04:16.454 --> 00:04:18.620
the probability of getting
it would be really small,

00:04:18.620 --> 00:04:20.839
and maybe we'll reject
the null hypothesis.

00:04:20.839 --> 00:04:24.530
So let's just figure out how
we can get the expected number.

00:04:24.530 --> 00:04:26.410
So we're assuming
the herbs do nothing.

00:04:26.410 --> 00:04:27.940
So if the herbs do
nothing, then we

00:04:27.940 --> 00:04:30.880
can just figure out that
this whole population just

00:04:30.880 --> 00:04:32.570
had nothing happen to them.

00:04:32.570 --> 00:04:33.870
These herbs were useless.

00:04:33.870 --> 00:04:36.555
And so we can use this
population sample--

00:04:36.555 --> 00:04:38.180
or I shouldn't call
it the population--

00:04:38.180 --> 00:04:40.120
we should use this
sample right here

00:04:40.120 --> 00:04:42.610
to figure out the
expected number of people

00:04:42.610 --> 00:04:44.260
who would get sick or not sick.

00:04:44.260 --> 00:04:50.210
And so over here, we have 80
out of 380 did not get sick.

00:04:50.210 --> 00:04:52.662
And I want to be careful, I
just said the word population,

00:04:52.662 --> 00:04:54.370
but we haven't sampled
the whole universe

00:04:54.370 --> 00:04:55.661
of all people taking this herb.

00:04:55.661 --> 00:04:57.300
This is a sample.

00:04:57.300 --> 00:04:58.860
So I don't want to confuse you.

00:04:58.860 --> 00:05:01.470
I was using population in more
of the conversational sense

00:05:01.470 --> 00:05:02.630
than the statistical sense.

00:05:02.630 --> 00:05:06.680
But anyway, of our sample--
and we're using all of the data

00:05:06.680 --> 00:05:09.191
because we're assuming
there's no difference.

00:05:09.191 --> 00:05:11.440
We might as well just use
the total data to figure out

00:05:11.440 --> 00:05:13.540
the expected frequency
of getting sick and not

00:05:13.540 --> 00:05:14.200
getting sick.

00:05:14.200 --> 00:05:17.780
So 80 divided by 380
did not get sick.

00:05:17.780 --> 00:05:20.130
And that's 21%.

00:05:20.130 --> 00:05:22.360
21% did not get sick.

00:05:22.360 --> 00:05:24.690
So let me write that over here.

00:05:24.690 --> 00:05:28.760
So 21, and that's
21% of the total,

00:05:28.760 --> 00:05:33.830
and then this would be 79% if
we just subtract 1 minus 21.

00:05:33.830 --> 00:05:38.620
We could divide 300 by 380,
and we should get 79% as well.

00:05:38.620 --> 00:05:40.810
So you would expect--
one would expect--

00:05:40.810 --> 00:05:45.800
that 21% of your total,
based on the total sample

00:05:45.800 --> 00:05:48.700
right over here, that our
best guess is that 21% percent

00:05:48.700 --> 00:05:52.560
should be getting sick and 79%
should not be getting sick.

00:05:52.560 --> 00:05:54.740
So let's look at it for
each of these groups.

00:05:54.740 --> 00:05:58.880
If we assume that 21%
of these 120 people

00:05:58.880 --> 00:06:00.620
should have gotten
sick, what would

00:06:00.620 --> 00:06:02.740
have been the expected
value right over here?

00:06:02.740 --> 00:06:08.630
So let's just multiply
21% times 120.

00:06:08.630 --> 00:06:12.340
So let's just multiply
that times 120.

00:06:12.340 --> 00:06:15.320
That gets us to 25 point--
I'll just round it--

00:06:15.320 --> 00:06:18.660
25.3 people should
have gotten sick.

00:06:18.660 --> 00:06:21.360
So the expected-- so let
me write it over here,

00:06:21.360 --> 00:06:28.660
I'll do expected in yellow-- so
the expected right over here.

00:06:28.660 --> 00:06:32.010
If you assume that 21% of each
group should have gotten sick

00:06:32.010 --> 00:06:37.000
is that you would have expected
25.3 people to get sick

00:06:37.000 --> 00:06:39.260
in group one, in herb one group.

00:06:39.260 --> 00:06:41.540
And then the remainder
will not get sick.

00:06:41.540 --> 00:06:45.260
So let's just subtract
or I could actually

00:06:45.260 --> 00:06:48.620
multiply 79% times 120, either
one of those would be good.

00:06:48.620 --> 00:06:58.680
But let me just take 120 minus
25.3, and then I get 94.7.

00:06:58.680 --> 00:07:04.070
So you would have expected
94.7 to not get sick.

00:07:04.070 --> 00:07:05.290
So this is expected again.

00:07:09.580 --> 00:07:12.580
94.7 to not get sick.

00:07:12.580 --> 00:07:15.020
And now let's do that
for each of these groups.

00:07:15.020 --> 00:07:16.730
So once again, group
two, you would've

00:07:16.730 --> 00:07:19.250
expected 21% to get sick.

00:07:19.250 --> 00:07:24.550
21% of the total people in
that group, so that's 140,

00:07:24.550 --> 00:07:26.390
so that's 29.4.

00:07:26.390 --> 00:07:30.080
And then the remainder--
let's see, 140 minus 29.4--

00:07:30.080 --> 00:07:31.270
should not have gotten sick.

00:07:31.270 --> 00:07:34.530
So that gets us this right here.

00:07:34.530 --> 00:07:41.430
We have 29.4 should have gotten
sick if the herbs did nothing.

00:07:41.430 --> 00:07:43.240
And then, over
here, we would have

00:07:43.240 --> 00:07:49.145
110.6 should not
have gotten sick.

00:07:49.145 --> 00:07:50.270
And these are pretty close.

00:07:50.270 --> 00:07:51.280
So, just looking
at the numbers, it

00:07:51.280 --> 00:07:53.230
looks this herb doesn't
do too much relative

00:07:53.230 --> 00:07:56.240
to the total, all of
the groups combined.

00:07:56.240 --> 00:07:58.757
And then in the placebo
group, let's see what happens.

00:07:58.757 --> 00:07:59.715
Let's see what happens.

00:08:02.640 --> 00:08:07.960
We expect 21% to get sick,
21% of our group of 120.

00:08:07.960 --> 00:08:12.250
So it's 25.2.

00:08:12.250 --> 00:08:14.110
So this right over here.

00:08:14.110 --> 00:08:18.315
And actually, this should be 25
point-- since we're rounding,

00:08:18.315 --> 00:08:20.440
actually, these will be
the same number over here--

00:08:20.440 --> 00:08:23.800
so I said 21%, but it's 21 point
something something something.

00:08:23.800 --> 00:08:25.810
The group sizes
are the same, so we

00:08:25.810 --> 00:08:27.870
should expect the same
proportion to get sick.

00:08:27.870 --> 00:08:31.560
So I'll say 25.3 just
to make it consistent.

00:08:31.560 --> 00:08:33.659
The reason why I
got 25.2 just now

00:08:33.659 --> 00:08:36.500
is because I lost some of the
trailing decimals over here.

00:08:36.500 --> 00:08:38.130
But since I had
them over here, I'm

00:08:38.130 --> 00:08:40.010
going to use them
over here as well.

00:08:40.010 --> 00:08:43.130
And then over here
in this group,

00:08:43.130 --> 00:08:49.707
you would expect
94.7 to get sick.

00:08:49.707 --> 00:08:51.540
So if you just actually
relied on this data,

00:08:51.540 --> 00:08:54.270
it looks like herb two is
actually, to some degree,

00:08:54.270 --> 00:08:59.380
even worse than the-- oh.

00:08:59.380 --> 00:09:00.520
No, no, I take that back.

00:09:00.520 --> 00:09:02.650
It's not worse because you would
have expected a small number,

00:09:02.650 --> 00:09:03.910
and a lot of people
got sick here.

00:09:03.910 --> 00:09:04.911
So this is the placebo--

00:09:04.911 --> 00:09:06.951
Well anyway, we don't want
to make judgments just

00:09:06.951 --> 00:09:07.990
staring at the numbers.

00:09:07.990 --> 00:09:11.200
Let's figure out our
chi-square statistic.

00:09:11.200 --> 00:09:14.620
And to do that, let's
get our statistic,

00:09:14.620 --> 00:09:16.990
our chi-square statistic.

00:09:16.990 --> 00:09:19.030
I'll write it like
this, maybe, for fun.

00:09:19.030 --> 00:09:23.400
Or maybe I'll write it as a
big X because it's really,

00:09:23.400 --> 00:09:25.574
this random variables
distribution,

00:09:25.574 --> 00:09:27.365
is approximately a
chi-square distribution.

00:09:27.365 --> 00:09:28.836
So I'll write it like that.

00:09:28.836 --> 00:09:31.460
And, well, we'll talk about the
degrees of freedom in a second.

00:09:31.460 --> 00:09:32.640
Actually, let me write
it with the curly X,

00:09:32.640 --> 00:09:34.556
just so you see that
some people write it with

00:09:34.556 --> 00:09:36.100
the chi instead of the X.

00:09:36.100 --> 00:09:38.930
So our chi-square
statistic over here.

00:09:38.930 --> 00:09:41.960
We're literally just going
to find the squared distance

00:09:41.960 --> 00:09:44.360
between the observed
and expected.

00:09:44.360 --> 00:09:46.150
And then divide it
by the expected.

00:09:46.150 --> 00:09:58.630
So it's going to be 20
minus 25.3 squared over 25.3

00:09:58.630 --> 00:10:06.580
plus 30 minus 29.4
squared over 29.4-- I'm

00:10:06.580 --> 00:10:14.010
going to run out of space--
plus 30 minus 25.3 squared

00:10:14.010 --> 00:10:16.149
over 25.3.

00:10:16.149 --> 00:10:18.190
And then I'm going to have
to do these over here,

00:10:18.190 --> 00:10:19.950
so let me just continue it.

00:10:19.950 --> 00:10:21.626
You could ignore
this H1 over here.

00:10:21.626 --> 00:10:29.430
So plus 100 minus
94.7 squared over

00:10:29.430 --> 00:10:32.760
94.7 plus-- I think you
see where this is going--

00:10:32.760 --> 00:10:39.950
110 minus 110.6
squared over 110.6.

00:10:39.950 --> 00:10:49.310
And then, finally,
plus 90 minus 94.7--

00:10:49.310 --> 00:10:51.800
let me scroll to the
right a little bit--

00:10:51.800 --> 00:10:55.219
squared, all of that over 94.7.

00:10:55.219 --> 00:10:56.760
So let me just get
the calculator out

00:10:56.760 --> 00:10:58.225
to calculate this.

00:10:58.225 --> 00:11:00.160
Take a little bit of time.

00:11:00.160 --> 00:11:02.330
So we have-- I have to
type on the calculator

00:11:02.330 --> 00:11:11.620
for these parentheses-- so
we have 20 minus 25.3 squared

00:11:11.620 --> 00:11:22.790
divided by 25.3 plus, open
parentheses, 30 minus 29.4

00:11:22.790 --> 00:11:37.710
squared divided by 29.4 plus,
open parentheses, 30 minus 25.3

00:11:37.710 --> 00:11:42.590
squared divided
by 25.3-- halfway

00:11:42.590 --> 00:11:47.320
there-- plus 100,
open parentheses,

00:11:47.320 --> 00:11:53.330
this is the tedious part,
100 minus 94.7 squared

00:11:53.330 --> 00:12:00.695
divided by 94.7 plus 110
minus-- I'll let you type

00:12:00.695 --> 00:12:02.570
it out, we can do a lot
of these in our head,

00:12:02.570 --> 00:12:09.000
but let me just do it--
110 minus 110.6 squared

00:12:09.000 --> 00:12:14.090
divided by 110.6-- and
then last one, homestretch,

00:12:14.090 --> 00:12:16.340
assuming we haven't
made any mistakes--

00:12:16.340 --> 00:12:25.830
we have 90 minus 94.7
squared divided by 94.7.

00:12:25.830 --> 00:12:27.990
And let's see what we get.

00:12:27.990 --> 00:12:34.130
We get 2.528, so let's
just say it's 2.53.

00:12:34.130 --> 00:12:38.500
So our chi-square
statistic-- always

00:12:38.500 --> 00:12:42.680
have trouble saying that-- our
chi-square statistic, assuming

00:12:42.680 --> 00:12:51.860
the null hypothesis is
correct, is equal to 2.53.

00:12:51.860 --> 00:12:53.360
Now, the next
thing we have to do

00:12:53.360 --> 00:12:55.180
is figure out the
degrees of freedom

00:12:55.180 --> 00:12:57.940
that we had in calculating
the chi-square statistic.

00:12:57.940 --> 00:12:59.440
And I'll give you
the rule of thumb,

00:12:59.440 --> 00:13:01.439
and I'll give you a little
bit of a sense of why

00:13:01.439 --> 00:13:04.220
this is the rule of thumb for
a contingency table like this.

00:13:04.220 --> 00:13:06.178
And in the future, we'll
talk a little bit more

00:13:06.178 --> 00:13:08.490
deeply about degrees of freedom.

00:13:08.490 --> 00:13:10.770
So the rule of thumb
for a contingency table

00:13:10.770 --> 00:13:14.270
is you have the number of
rows, so you have rows,

00:13:14.270 --> 00:13:16.070
and then you have your
number of columns.

00:13:16.070 --> 00:13:18.236
So here we have two rows,
and we have three columns.

00:13:18.236 --> 00:13:19.680
You don't count the totals.

00:13:19.680 --> 00:13:21.600
So you have three
columns over here.

00:13:21.600 --> 00:13:27.420
And the degrees of freedom,
and this is the rule of thumb,

00:13:27.420 --> 00:13:30.420
the degrees of freedom
for your contingency table

00:13:30.420 --> 00:13:33.820
is going to be the number of
rows minus 1 times the number

00:13:33.820 --> 00:13:35.450
of columns minus 1.

00:13:35.450 --> 00:13:39.750
In our situation, we have
2 rows and 3 columns.

00:13:39.750 --> 00:13:43.550
So it's going to be 2
minus 1 times 3 minus 1.

00:13:43.550 --> 00:13:47.830
So it's going to be 2
minus 1 times 3 minus 1,

00:13:47.830 --> 00:13:51.350
which is just 1
times 2, which is 2.

00:13:51.350 --> 00:13:53.539
We have 2 degrees of freedom.

00:13:53.539 --> 00:13:55.080
Now, the reason that
that should make

00:13:55.080 --> 00:13:56.663
a little bit of
intuitive sense, we'll

00:13:56.663 --> 00:13:58.860
talk about this in more
depth in the future,

00:13:58.860 --> 00:14:02.324
is that if you assume
that you know the totals.

00:14:02.324 --> 00:14:04.240
So let's just assume
that you know the totals.

00:14:04.240 --> 00:14:07.430
So if you know all of this
information over here,

00:14:07.430 --> 00:14:10.771
if you know the total
information-- or actually,

00:14:10.771 --> 00:14:13.020
if you knew the parameters
of the population as well--

00:14:13.020 --> 00:14:14.603
but if you know the
total information,

00:14:14.603 --> 00:14:16.390
and if you know
this information,

00:14:16.390 --> 00:14:22.080
or if you know r minus 1 of
the information in the rows,

00:14:22.080 --> 00:14:23.630
the last one can
be figured out just

00:14:23.630 --> 00:14:25.490
by subtracting from the total.

00:14:25.490 --> 00:14:28.270
So for example, in this
situation, if you know this,

00:14:28.270 --> 00:14:29.760
you can easily figure out this.

00:14:29.760 --> 00:14:33.020
This is not new information,
it's just the total minus 20.

00:14:33.020 --> 00:14:35.340
Same thing, if you know
this one right over here,

00:14:35.340 --> 00:14:37.760
this one over here is
not new information.

00:14:37.760 --> 00:14:41.790
And similarly, if you know
these two, this guy over here

00:14:41.790 --> 00:14:43.170
isn't new information.

00:14:43.170 --> 00:14:45.090
You could always
just calculate him

00:14:45.090 --> 00:14:46.720
based on the total
and everything else.

00:14:46.720 --> 00:14:49.350
So that's the sense of
why our degrees of freedom

00:14:49.350 --> 00:14:54.510
are the columns minus 1
times the rows minus 1.

00:14:54.510 --> 00:14:56.270
But anyway, so our
chi-square statistic

00:14:56.270 --> 00:14:59.140
has 2 degrees of freedom.

00:14:59.140 --> 00:15:02.160
So what we have to do is
remember our alpha value-- let

00:15:02.160 --> 00:15:03.610
me get it up here,
we had it right

00:15:03.610 --> 00:15:05.860
over here-- our significance
level that we care about,

00:15:05.860 --> 00:15:07.550
our alpha value is 10%.

00:15:07.550 --> 00:15:09.380
Let me rewrite it over here.

00:15:09.380 --> 00:15:11.640
So our alpha is 10%.

00:15:11.640 --> 00:15:13.960
So what we're going
to do is figure out

00:15:13.960 --> 00:15:18.820
what is our critical
chi-square statistic that

00:15:18.820 --> 00:15:20.340
gives us an alpha of 10%.

00:15:20.340 --> 00:15:22.454
If this is more
extreme than that--

00:15:22.454 --> 00:15:23.870
if the probability
of getting this

00:15:23.870 --> 00:15:25.870
is even less than that
critical statistic--

00:15:25.870 --> 00:15:27.820
it'll be less than
10%, and we'll

00:15:27.820 --> 00:15:29.580
reject the null hypothesis.

00:15:29.580 --> 00:15:31.400
If it's not more
extreme, then we

00:15:31.400 --> 00:15:32.887
won't reject the
null hypothesis.

00:15:32.887 --> 00:15:34.470
So what we need to
do is to figure out

00:15:34.470 --> 00:15:37.860
with the chi-square distribution
and 2 degrees of freedom, what

00:15:37.860 --> 00:15:40.450
is our critical
chi-square statistic.

00:15:40.450 --> 00:15:42.110
So let's just go back.

00:15:42.110 --> 00:15:43.870
So we have 2 degrees of freedom.

00:15:49.250 --> 00:15:53.880
We care about a
significance level of 10%.

00:15:53.880 --> 00:16:00.530
So our critical
chi-square value is 4.60.

00:16:00.530 --> 00:16:02.510
So another way to
visualize this.

00:16:02.510 --> 00:16:05.210
If we look at the chi-square
distribution with 2 degrees

00:16:05.210 --> 00:16:09.570
of freedom, that's this
blue one over here,

00:16:09.570 --> 00:16:12.710
at a value of-- I'm trying
to pick a nice blue to use--

00:16:12.710 --> 00:16:16.920
at a critical value of 4.60.

00:16:16.920 --> 00:16:21.150
So 4.60-- this is 5-- so 4.60
will be right around here.

00:16:21.150 --> 00:16:27.550
At a critical value of
4.60, so this is 4.60,

00:16:27.550 --> 00:16:29.950
the probability of getting
something at least that

00:16:29.950 --> 00:16:34.300
extreme, so that extreme
or more extreme, is 10%.

00:16:34.300 --> 00:16:36.010
This is what we care about.

00:16:36.010 --> 00:16:38.830
Now, if the chi-square
statistic that we calculated

00:16:38.830 --> 00:16:41.350
falls into this
rejection region,

00:16:41.350 --> 00:16:43.890
then we're going to reject
the null hypothesis.

00:16:43.890 --> 00:16:50.340
But our chi-square
statistic is only 2.53.

00:16:50.340 --> 00:16:52.380
It is only 2.53.

00:16:52.380 --> 00:16:57.240
So it's sitting someplace right
over here is actually ours.

00:16:57.240 --> 00:17:01.180
So it's actually not
that crazy to get it

00:17:01.180 --> 00:17:04.530
if you assume the
null hypothesis.

00:17:04.530 --> 00:17:06.569
So based on our data
that we have right now,

00:17:06.569 --> 00:17:09.750
we cannot reject
the null hypothesis.

00:17:09.750 --> 00:17:13.069
So we don't know for a fact
that the herbs do nothing,

00:17:13.069 --> 00:17:17.060
but we can't say that they
do something based on this.

00:17:17.060 --> 00:17:18.819
So we're not going to reject it.

00:17:18.819 --> 00:17:20.420
We won't say 100%
that it's true,

00:17:20.420 --> 00:17:22.440
but we can't say that
we're rejecting it.

00:17:22.440 --> 00:17:25.239
So at least from
this point of view,

00:17:25.239 --> 00:17:27.030
it doesn't look like
the herbs did anything

00:17:27.030 --> 00:17:29.320
that would make us
believe that they're

00:17:29.320 --> 00:17:31.917
any different than each other.

00:17:31.917 --> 00:17:33.750
And one of the herbs
is obviously a placebo.

00:17:33.750 --> 00:17:37.070
So any different than a
placebo or each other.


Pearson's chi square test (goodness of fit)

WEBVTT
Kind: captions
Language: en

00:00:00.660 --> 00:00:02.614
I'm thinking about
buying a restaurant,

00:00:02.614 --> 00:00:04.030
so I go and ask
the current owner,

00:00:04.030 --> 00:00:06.920
what is the distribution
of the number of customers

00:00:06.920 --> 00:00:07.954
you get each day?

00:00:07.954 --> 00:00:09.870
And he says, oh, I've
already figure that out.

00:00:09.870 --> 00:00:11.372
And he gives me
this distribution

00:00:11.372 --> 00:00:13.830
over here, which essentially
says 10% of his customers come

00:00:13.830 --> 00:00:17.080
in on Monday, 10% on
Tuesday, 15% on Wednesday,

00:00:17.080 --> 00:00:18.250
so forth, and so on.

00:00:18.250 --> 00:00:19.570
They're closed on Sunday.

00:00:19.570 --> 00:00:21.760
So this is 100% of the
customers for a week.

00:00:21.760 --> 00:00:23.466
If you add that
up, you get 100%.

00:00:23.466 --> 00:00:25.090
I obviously am a
little bit suspicious,

00:00:25.090 --> 00:00:30.010
so I decide to see how good
this distribution that he's

00:00:30.010 --> 00:00:32.200
describing actually
fits observed data.

00:00:32.200 --> 00:00:35.406
So I actually observe the number
of customers, when they come in

00:00:35.406 --> 00:00:36.780
during the week,
and this is what

00:00:36.780 --> 00:00:38.750
I get from my observed data.

00:00:38.750 --> 00:00:42.820
So to figure out whether
I want to accept or reject

00:00:42.820 --> 00:00:44.480
his hypothesis right
here, I'm going

00:00:44.480 --> 00:00:47.140
to do a little bit
of a hypothesis test.

00:00:47.140 --> 00:00:57.950
So I'll make the null hypothesis
that the owner's distribution--

00:00:57.950 --> 00:01:00.220
so that's this thing
right here-- is correct.

00:01:03.050 --> 00:01:06.670
And then the
alternative hypothesis

00:01:06.670 --> 00:01:09.500
is going to be that
it is not correct,

00:01:09.500 --> 00:01:11.600
that it is not a
correct distribution,

00:01:11.600 --> 00:01:15.310
that I should not feel
reasonably OK relying on this.

00:01:15.310 --> 00:01:17.410
It's not the correct--
I should reject

00:01:17.410 --> 00:01:19.070
the owner's distribution.

00:01:19.070 --> 00:01:24.340
And I want to do this with
a significance level of 5%.

00:01:27.001 --> 00:01:28.500
Or another way of
thinking about it,

00:01:28.500 --> 00:01:31.420
I'm going to calculate a
statistic based on this data

00:01:31.420 --> 00:01:32.060
right here.

00:01:32.060 --> 00:01:34.520
And it's going to be
chi-square statistic.

00:01:34.520 --> 00:01:37.030
Or another way to view
it is it that statistic

00:01:37.030 --> 00:01:39.560
that I'm going to
calculate has approximately

00:01:39.560 --> 00:01:41.720
a chi-square distribution.

00:01:41.720 --> 00:01:44.009
And given that it does have
a chi-square distribution

00:01:44.009 --> 00:01:45.800
with a certain number
of degrees of freedom

00:01:45.800 --> 00:01:48.930
and we're going to calculate
that, what I want to see

00:01:48.930 --> 00:01:51.700
is the probability of
getting this result,

00:01:51.700 --> 00:01:55.320
or getting a result like
this or a result more extreme

00:01:55.320 --> 00:01:57.159
less than 5%.

00:01:57.159 --> 00:01:59.700
If the probability of getting
a result like this or something

00:01:59.700 --> 00:02:03.180
less likely than
this is less than 5%,

00:02:03.180 --> 00:02:06.780
then I'm going to reject
the null hypothesis, which

00:02:06.780 --> 00:02:10.919
is essentially just rejecting
the owner's distribution.

00:02:10.919 --> 00:02:13.540
If I don't get
that, if I say, hey,

00:02:13.540 --> 00:02:16.990
the probability of getting
a chi-square statistic that

00:02:16.990 --> 00:02:22.070
is this extreme or more
is greater than my alpha,

00:02:22.070 --> 00:02:24.570
than my significance level,
then I'm not going to reject it.

00:02:24.570 --> 00:02:26.236
I'm going to say,
well, I have no reason

00:02:26.236 --> 00:02:28.200
to really assume
that he's lying.

00:02:28.200 --> 00:02:29.670
So let's do that.

00:02:29.670 --> 00:02:32.790
So to calculate the chi-square
statistic, what I'm going to do

00:02:32.790 --> 00:02:36.361
is-- so here we're assuming
the owner's distribution is

00:02:36.361 --> 00:02:36.860
correct.

00:02:41.360 --> 00:02:43.330
So assuming the
owner's distribution

00:02:43.330 --> 00:02:47.790
was correct, what would have
been the expected observed?

00:02:47.790 --> 00:02:49.680
So we have expected
percentage here,

00:02:49.680 --> 00:02:51.639
but what would have been
the expected observed?

00:02:51.639 --> 00:02:52.971
So let me write this right here.

00:02:52.971 --> 00:02:53.700
Expected.

00:02:53.700 --> 00:02:56.950
I'll add another row, Expected.

00:02:56.950 --> 00:02:59.970
So we would have expected
10% of the total customers

00:02:59.970 --> 00:03:01.490
in that week to
come in on Monday,

00:03:01.490 --> 00:03:03.160
10% of the total
customers of that week

00:03:03.160 --> 00:03:05.900
to come in on Tuesday, 15%
to come in on Wednesday.

00:03:05.900 --> 00:03:08.070
Now to figure out what
the actual number is,

00:03:08.070 --> 00:03:10.540
we need to figure out the
total number of customers.

00:03:10.540 --> 00:03:14.100
So let's add up these
numbers right here.

00:03:14.100 --> 00:03:17.790
So we have-- I'll get
the calculator out.

00:03:17.790 --> 00:03:26.600
So we have 30 plus 14 plus
34 plus 45 plus 57 plus 20.

00:03:26.600 --> 00:03:28.470
So there's a total
of 200 customers who

00:03:28.470 --> 00:03:30.830
came into the
restaurant that week.

00:03:30.830 --> 00:03:32.410
So let me write this down.

00:03:32.410 --> 00:03:37.850
So this is equal to-- so I
wrote the total over here.

00:03:37.850 --> 00:03:38.960
Ignore this right here.

00:03:38.960 --> 00:03:41.270
I had 200 customers
come in for the week.

00:03:41.270 --> 00:03:43.530
So what was the expected
number on Monday?

00:03:43.530 --> 00:03:46.700
Well, on Monday, we would
have expected 10% of the 200

00:03:46.700 --> 00:03:47.330
to come in.

00:03:47.330 --> 00:03:51.020
So this would have been 20
customers, 10% times 200.

00:03:51.020 --> 00:03:52.670
On Tuesday, another 10%.

00:03:52.670 --> 00:03:54.860
So we would have
expected 20 customers.

00:03:54.860 --> 00:03:59.210
Wednesday, 15% of 200,
that's 30 customers.

00:03:59.210 --> 00:04:03.310
On Thursday, we would have
expected 20% of 200 customers,

00:04:03.310 --> 00:04:05.430
so that would have
been 40 customers.

00:04:05.430 --> 00:04:09.080
Then on Friday, 30%, that
would have been 60 customers.

00:04:09.080 --> 00:04:10.900
And then on Friday 15% again.

00:04:10.900 --> 00:04:14.110
15% of 200 would have
been 30 customers.

00:04:14.110 --> 00:04:16.110
So if this distribution
is correct,

00:04:16.110 --> 00:04:20.690
this is the actual number
that I would have expected.

00:04:20.690 --> 00:04:23.700
Now to calculate
chi-square statistic,

00:04:23.700 --> 00:04:26.510
we essentially just take--
let me just show it to you,

00:04:26.510 --> 00:04:28.700
and instead of
writing chi, I'm going

00:04:28.700 --> 00:04:30.010
to write capital X squared.

00:04:30.010 --> 00:04:33.000
Sometimes someone will write the
actual Greek letter chi here.

00:04:33.000 --> 00:04:35.692
But I'll write the
x squared here.

00:04:35.692 --> 00:04:36.900
And let me write it this way.

00:04:36.900 --> 00:04:45.462
This is our
chi-square statistic,

00:04:45.462 --> 00:04:47.920
but I'm going to write it with
a capital X instead of a chi

00:04:47.920 --> 00:04:50.042
because this is going
to have approximately

00:04:50.042 --> 00:04:52.190
a chi-squared distribution.

00:04:52.190 --> 00:04:53.640
I can't assume
that it's exactly,

00:04:53.640 --> 00:04:56.490
so this is where we're dealing
with approximations right here.

00:04:56.490 --> 00:04:59.380
But it's fairly
straightforward to calculate.

00:04:59.380 --> 00:05:01.230
For each of the days,
we take the difference

00:05:01.230 --> 00:05:03.420
between the observed
and expected.

00:05:03.420 --> 00:05:08.340
So it's going to
be 30 minus 20--

00:05:08.340 --> 00:05:12.010
I'll do the first one
color coded-- squared

00:05:12.010 --> 00:05:14.340
divided by the expected.

00:05:14.340 --> 00:05:16.200
So we're essentially
taking the square

00:05:16.200 --> 00:05:18.650
of almost you could kind of
do the error between what

00:05:18.650 --> 00:05:21.727
we observed and expected or
the difference between what

00:05:21.727 --> 00:05:24.060
we observed and expect, and
we're kind of normalizing it

00:05:24.060 --> 00:05:25.900
by the expected right over here.

00:05:25.900 --> 00:05:28.210
But we want to take the
sum of all of these.

00:05:28.210 --> 00:05:30.950
So I'll just do all
of those in yellow.

00:05:30.950 --> 00:05:44.570
So plus 14 minus 20 squared
over 20 plus 34 minus 30 squared

00:05:44.570 --> 00:05:54.300
over 30 plus-- I'll continue
over here-- 45 minus 40 squared

00:05:54.300 --> 00:06:04.550
over 40 plus 57 minus
60 squared over 60,

00:06:04.550 --> 00:06:12.710
and then finally, plus 20
minus 30 squared over 30.

00:06:12.710 --> 00:06:14.930
I just took the observed
minus the expected

00:06:14.930 --> 00:06:16.290
squared over the expected.

00:06:16.290 --> 00:06:17.890
I took the sum of
it, and this is

00:06:17.890 --> 00:06:20.370
what gives us our
chi-square statistic.

00:06:20.370 --> 00:06:24.080
Now let's just calculate what
this number is going to be.

00:06:24.080 --> 00:06:26.671
So this is going to be equal
to-- I'll do it over here

00:06:26.671 --> 00:06:27.920
so you don't run out of space.

00:06:27.920 --> 00:06:29.860
So we'll do this a new color.

00:06:29.860 --> 00:06:30.820
We'll do it in orange.

00:06:30.820 --> 00:06:34.140
This is going to be
equal to 30 minus 20

00:06:34.140 --> 00:06:40.752
is 10 squared, which is 100
divided by 20, which is 5.

00:06:40.752 --> 00:06:43.210
I might not be able to do all
of them in my head like this.

00:06:43.210 --> 00:06:45.090
Plus, actually, let me
just write it this way

00:06:45.090 --> 00:06:47.830
just so you can
see what I'm doing.

00:06:47.830 --> 00:06:53.250
This right here is 100
over 20 plus 14 minus 20

00:06:53.250 --> 00:06:56.170
is negative 6 squared
is positive 36.

00:06:56.170 --> 00:06:59.570
So plus 36 over 20.

00:06:59.570 --> 00:07:04.040
Plus 34 minus 30 is
4, squared is 16.

00:07:04.040 --> 00:07:07.140
So plus 16 over 30.

00:07:07.140 --> 00:07:11.230
Plus 45 minus 40
is 5 squared is 25.

00:07:11.230 --> 00:07:14.550
So plus 25 over 40.

00:07:14.550 --> 00:07:18.050
Plus the difference
here is 3 squared is 9,

00:07:18.050 --> 00:07:20.370
so it's 9 over 60.

00:07:20.370 --> 00:07:27.442
Plus we have a difference of
10 squared is plus 100 over 30.

00:07:27.442 --> 00:07:29.900
And this is equal to-- and I'll
just get the calculator out

00:07:29.900 --> 00:07:36.500
for this-- this is
equal to, we have

00:07:36.500 --> 00:07:41.560
100 divided by 20
plus 36 divided by 20

00:07:41.560 --> 00:07:49.300
plus 16 divided by 30
plus 25 divided by 40

00:07:49.300 --> 00:08:02.260
plus 9 divided by 60 plus 100
divided by 30 gives us 11.44.

00:08:02.260 --> 00:08:03.390
So let me write that down.

00:08:03.390 --> 00:08:10.290
So this right here
is going to be 11.44.

00:08:10.290 --> 00:08:12.260
This is my chi-square
statistic, or we

00:08:12.260 --> 00:08:13.890
could call it a big
capital X squared.

00:08:13.890 --> 00:08:16.970
Sometimes you'll have it
written as a chi-square,

00:08:16.970 --> 00:08:19.990
but this statistic is
going to have approximately

00:08:19.990 --> 00:08:21.780
a chi-square distribution.

00:08:21.780 --> 00:08:24.430
Anyway, with that
said, let's figure out,

00:08:24.430 --> 00:08:28.000
if we assume that it has roughly
a chi-square distribution, what

00:08:28.000 --> 00:08:32.720
is the probability of getting a
result this extreme or at least

00:08:32.720 --> 00:08:35.520
this extreme, I guess is another
way of thinking about it.

00:08:35.520 --> 00:08:39.750
Or another way of saying, is
this a more extreme result

00:08:39.750 --> 00:08:42.450
than the critical
chi-square value

00:08:42.450 --> 00:08:45.491
that there's a 5% chance of
getting a result that extreme?

00:08:45.491 --> 00:08:46.490
So let's do it that way.

00:08:46.490 --> 00:08:48.610
Let's figure out the
critical chi-square value.

00:08:48.610 --> 00:08:50.420
And if this is more
extreme than that,

00:08:50.420 --> 00:08:53.280
then we will reject
our null hypothesis.

00:08:53.280 --> 00:08:56.900
So let's figure out our
critical chi-square values.

00:08:56.900 --> 00:08:58.207
So we have an alpha of 5%.

00:08:58.207 --> 00:09:00.290
And actually the other
thing we have to figure out

00:09:00.290 --> 00:09:03.080
is the degrees of freedom.

00:09:03.080 --> 00:09:07.470
The degrees of freedom, we're
taking one, two, three, four,

00:09:07.470 --> 00:09:09.380
five, six sums, so
you might be tempted

00:09:09.380 --> 00:09:11.240
to say the degrees
of freedom are six.

00:09:11.240 --> 00:09:12.960
But one thing to
realize is that if you

00:09:12.960 --> 00:09:15.310
had all of this
information over here,

00:09:15.310 --> 00:09:19.650
you could actually figure out
this last piece of information,

00:09:19.650 --> 00:09:21.720
so you actually have
five degrees of freedom.

00:09:21.720 --> 00:09:24.390
When you have just kind of
n data points like this,

00:09:24.390 --> 00:09:26.950
and you're measuring kind of
the observed versus expected,

00:09:26.950 --> 00:09:29.090
your degrees of freedom
are going to be n minus 1,

00:09:29.090 --> 00:09:31.336
because you could figure
out that nth data point just

00:09:31.336 --> 00:09:32.960
based on everything
else that you have,

00:09:32.960 --> 00:09:34.970
all of the other information.

00:09:34.970 --> 00:09:37.255
So our degrees of freedom
here are going to be 5.

00:09:37.255 --> 00:09:39.540
It's n minus 1.

00:09:39.540 --> 00:09:43.010
So our significance level is 5%.

00:09:43.010 --> 00:09:48.230
And our degrees of freedom is
also going to be equal to 5.

00:09:48.230 --> 00:09:51.080
So let's look at our
chi-square distribution.

00:09:51.080 --> 00:09:52.690
We have a degree
of freedom of 5.

00:09:56.450 --> 00:09:59.320
We have a significance
level of 5%.

00:09:59.320 --> 00:10:03.914
And so the critical
chi-square value is 11.07.

00:10:03.914 --> 00:10:05.080
So let's go with this chart.

00:10:05.080 --> 00:10:07.340
So we have a
chi-squared distribution

00:10:07.340 --> 00:10:08.980
with a degree of freedom of 5.

00:10:08.980 --> 00:10:12.050
So that's this distribution
over here in magenta.

00:10:12.050 --> 00:10:16.500
And we care about a
critical value of 11.07.

00:10:16.500 --> 00:10:17.417
So this is right here.

00:10:17.417 --> 00:10:19.208
Oh, you actually even
can't see it on this.

00:10:19.208 --> 00:10:21.410
So if I were to keep drawing
this magenta thing all

00:10:21.410 --> 00:10:26.780
the way over here, if the
magenta line just kept going,

00:10:26.780 --> 00:10:28.630
over here, you'd have 8.

00:10:28.630 --> 00:10:30.010
Over here you'd have 10.

00:10:30.010 --> 00:10:32.000
Over here, you'd have 12.

00:10:32.000 --> 00:10:36.110
11.07 is maybe some
place right over there.

00:10:36.110 --> 00:10:38.390
So what it's saying
is the probability

00:10:38.390 --> 00:10:50.219
of getting a result at least
as extreme as 11.07 is 5%.

00:10:50.219 --> 00:10:51.510
So we could write it even here.

00:10:51.510 --> 00:10:58.370
Our critical chi-square value is
equal to-- we just saw-- 11.07.

00:10:58.370 --> 00:11:00.000
Let me look at the chart again.

00:11:00.000 --> 00:11:07.170
11.07.

00:11:07.170 --> 00:11:09.260
The result we got
for our statistic

00:11:09.260 --> 00:11:13.450
is even less likely than that.

00:11:13.450 --> 00:11:16.140
The probability is less
than our significance level.

00:11:16.140 --> 00:11:19.120
So then we are going to reject.

00:11:19.120 --> 00:11:21.180
So the probability
of getting that is--

00:11:21.180 --> 00:11:26.740
let me put it this
way-- 11.44 is

00:11:26.740 --> 00:11:31.180
more extreme than our
critical chi-square level.

00:11:31.180 --> 00:11:35.720
So it's very unlikely that
this distribution is true.

00:11:35.720 --> 00:11:41.880
So we will reject
what he's telling us.

00:11:41.880 --> 00:11:43.660
We will reject
this distribution.

00:11:43.660 --> 00:11:48.000
It's not a good fit based
on this significance level.


Chi-square distribution introduction

WEBVTT
Kind: captions
Language: en

00:00:00.619 --> 00:00:02.410
In this video, we'll
just talk a little bit

00:00:02.410 --> 00:00:10.210
about what the chi-square
distribution is, sometimes

00:00:10.210 --> 00:00:12.387
called the chi-squared
distribution.

00:00:12.387 --> 00:00:13.970
And then in the next
few videos, we'll

00:00:13.970 --> 00:00:17.820
actually use it to
really test how well

00:00:17.820 --> 00:00:20.820
theoretical distributions
explain observed ones,

00:00:20.820 --> 00:00:23.260
or how good a fit
observed results

00:00:23.260 --> 00:00:25.940
are for theoretical
distributions.

00:00:25.940 --> 00:00:27.890
So let's just think
about it a little bit.

00:00:27.890 --> 00:00:29.640
So let's say I have
some random variables.

00:00:29.640 --> 00:00:31.540
And each of them
are independent,

00:00:31.540 --> 00:00:35.720
standard, normal, normally
distributed random variables.

00:00:35.720 --> 00:00:37.690
So let me just remind
you what that means.

00:00:37.690 --> 00:00:41.450
So let's say I have the random
variable X. If X is normally

00:00:41.450 --> 00:00:43.590
distributed, we
could write that X

00:00:43.590 --> 00:00:49.480
is a normal random
variable with a mean of 0

00:00:49.480 --> 00:00:51.940
and a variance of 1.

00:00:51.940 --> 00:00:56.290
Or you could say that
the expected value of X,

00:00:56.290 --> 00:00:59.960
is equal to 0, or
in that the variance

00:00:59.960 --> 00:01:03.160
of our random variable
X is equal to 1.

00:01:03.160 --> 00:01:06.620
Or just to visualize
it is that, when

00:01:06.620 --> 00:01:09.310
we take an instantiation
of this very variable,

00:01:09.310 --> 00:01:12.770
we're sampling from a
normal distribution,

00:01:12.770 --> 00:01:14.560
a standardized normal
distribution that

00:01:14.560 --> 00:01:15.850
looks like this.

00:01:15.850 --> 00:01:19.910
Mean of 0 and then a variance
of 1, which would also mean,

00:01:19.910 --> 00:01:22.080
of course, a standard
deviation of 1.

00:01:22.080 --> 00:01:26.810
So that could be the standard
deviation, or the variance,

00:01:26.810 --> 00:01:30.012
or the standard deviation,
that would be equal to 1.

00:01:30.012 --> 00:01:31.720
So a chi-square
distribution, if you just

00:01:31.720 --> 00:01:34.120
take one of these
random variables--

00:01:34.120 --> 00:01:35.740
and let me define it this way.

00:01:35.740 --> 00:01:38.360
Let me define a new
random variable.

00:01:38.360 --> 00:01:40.230
Let me define a
new random variable

00:01:40.230 --> 00:01:47.010
Q that is equal to--
you're essentially

00:01:47.010 --> 00:01:50.210
sampling from this the
standard normal distribution

00:01:50.210 --> 00:01:52.470
and then squaring
whatever number you got.

00:01:52.470 --> 00:01:59.360
So it is equal to this
random variable X squared.

00:02:03.100 --> 00:02:06.010
The distribution for this
random variable right

00:02:06.010 --> 00:02:09.699
here is going to be an example
of the chi-square distribution.

00:02:09.699 --> 00:02:11.740
Actually what we're going
to see in this video is

00:02:11.740 --> 00:02:14.900
that the chi-square, or the
chi-squared distribution

00:02:14.900 --> 00:02:16.710
is actually a set
of distributions

00:02:16.710 --> 00:02:18.870
depending on how
many sums you have.

00:02:18.870 --> 00:02:21.620
Right now, we only have
one random variable

00:02:21.620 --> 00:02:22.750
that we're squaring.

00:02:22.750 --> 00:02:25.670
So this is just one
of the examples.

00:02:25.670 --> 00:02:27.710
And we'll talk more
about them in a second.

00:02:27.710 --> 00:02:29.710
So this right
here, this we could

00:02:29.710 --> 00:02:35.410
write that Q is a chi-squared
distributed random variable.

00:02:35.410 --> 00:02:37.850
Or that we could use
this notation right here.

00:02:37.850 --> 00:02:40.780
Q is-- we could
write it like this.

00:02:40.780 --> 00:02:42.170
So this isn't an X anymore.

00:02:42.170 --> 00:02:43.910
This is the Greek
letter chi, although it

00:02:43.910 --> 00:02:49.210
looks a lot like a curvy X. So
it's a member of chi-squared.

00:02:49.210 --> 00:02:51.600
And since we're only
taking one sum over here--

00:02:51.600 --> 00:02:54.180
we're only taking the
sum of one independent,

00:02:54.180 --> 00:02:57.450
normally distributed, standard
or normally distributed

00:02:57.450 --> 00:03:02.060
variable, we say that this
only has 1 degree of freedom.

00:03:02.060 --> 00:03:03.270
And we write that over here.

00:03:03.270 --> 00:03:09.880
So this right here is
our degree of freedom.

00:03:09.880 --> 00:03:14.850
We have 1 degree of
freedom right over there.

00:03:14.850 --> 00:03:16.630
So let's call this Q1.

00:03:16.630 --> 00:03:19.150
Let's say I have
another random variable.

00:03:19.150 --> 00:03:21.820
Let's call this Q-- let me
do it in a different color.

00:03:21.820 --> 00:03:24.220
Let me do Q2 in blue.

00:03:24.220 --> 00:03:26.540
Let's say I have another
random variable, Q2,

00:03:26.540 --> 00:03:31.470
that is defined as-- let's say I
have one independent, standard,

00:03:31.470 --> 00:03:33.380
normally distributed variable.

00:03:33.380 --> 00:03:34.700
I'll call that X1.

00:03:34.700 --> 00:03:35.960
And I square it.

00:03:35.960 --> 00:03:44.030
And then I have another
independent, standard,

00:03:44.030 --> 00:03:47.240
normally distributed
variable, X2.

00:03:47.240 --> 00:03:48.115
And I square it.

00:03:48.115 --> 00:03:49.740
So you could imagine
both of these guys

00:03:49.740 --> 00:03:51.120
have distributions like this.

00:03:51.120 --> 00:03:52.120
And they're independent.

00:03:52.120 --> 00:03:57.220
So get to sample Q2,
you essentially sample

00:03:57.220 --> 00:04:01.990
X1 from this distribution,
square that value, sample X2

00:04:01.990 --> 00:04:05.370
from the same distribution,
essentially, square that value,

00:04:05.370 --> 00:04:06.840
and then add the two.

00:04:06.840 --> 00:04:08.420
And you're going to get Q2.

00:04:08.420 --> 00:04:12.280
This over here-- here we
would write-- so this is Q1.

00:04:12.280 --> 00:04:16.620
Q2 here, Q2 we would
write is a chi-squared,

00:04:16.620 --> 00:04:21.550
distributed random variable
with 2 degrees of freedom.

00:04:21.550 --> 00:04:22.050
Right here.

00:04:22.050 --> 00:04:27.180
2 degrees of freedom.

00:04:27.180 --> 00:04:29.910
And just to visualize
kind of the set

00:04:29.910 --> 00:04:33.990
of chi-squared distributions,
let's look at this over here.

00:04:33.990 --> 00:04:36.410
So this, I got this
off of Wikipedia.

00:04:36.410 --> 00:04:39.900
This shows us some of the
probability density functions

00:04:39.900 --> 00:04:42.270
for some of the
chi-square distributions.

00:04:42.270 --> 00:04:45.220
This first one over here,
for k of equal to 1,

00:04:45.220 --> 00:04:47.080
that's the degrees of freedom.

00:04:47.080 --> 00:04:49.200
So this is essentially our Q1.

00:04:49.200 --> 00:04:52.300
This is our probability
density function for Q1.

00:04:52.300 --> 00:04:54.270
And notice it really
spikes close to 0.

00:04:54.270 --> 00:04:55.470
And that makes sense.

00:04:55.470 --> 00:04:59.220
Because if you are
sampling just once

00:04:59.220 --> 00:05:02.380
from this standard
normal distribution,

00:05:02.380 --> 00:05:04.610
there's a very high
likelihood that you're

00:05:04.610 --> 00:05:06.609
going to get something
pretty close to 0.

00:05:06.609 --> 00:05:08.900
And then if you square
something close to 0-- remember,

00:05:08.900 --> 00:05:11.066
these are decimals, they're
going to be less than 1,

00:05:11.066 --> 00:05:14.120
pretty close to 0-- it's
going to become even smaller.

00:05:14.120 --> 00:05:17.470
So you have a high probability
of getting a very small value.

00:05:17.470 --> 00:05:19.380
You have high probabilities
of getting values

00:05:19.380 --> 00:05:23.290
less than some threshold,
this right here, less than,

00:05:23.290 --> 00:05:24.670
I guess, this is 1 right here.

00:05:24.670 --> 00:05:25.862
So the less than 1/2.

00:05:25.862 --> 00:05:27.320
And you have a very
low probability

00:05:27.320 --> 00:05:28.500
of getting a large number.

00:05:28.500 --> 00:05:34.017
I mean, to get a 4, you
would have to sample a 2

00:05:34.017 --> 00:05:34.975
from this distribution.

00:05:34.975 --> 00:05:39.830
And we know that 2 is--
actually it's 2 variances

00:05:39.830 --> 00:05:41.460
or 2 standard deviations
from the mean.

00:05:41.460 --> 00:05:42.870
So it's less likely.

00:05:42.870 --> 00:05:45.290
And actually that's to get a 4.

00:05:45.290 --> 00:05:46.910
So to get even
larger numbers are

00:05:46.910 --> 00:05:49.560
going to be even less likely.

00:05:49.560 --> 00:05:51.435
So that's why you see
this shape over here.

00:05:51.435 --> 00:05:53.060
Now when you have 2
degrees of freedom,

00:05:53.060 --> 00:05:54.960
it moderates a little bit.

00:05:54.960 --> 00:05:59.260
This is the shape,
this blue line right

00:05:59.260 --> 00:06:01.910
here is the shape of Q2.

00:06:01.910 --> 00:06:04.890
And notice you're a little
bit less likely to get values

00:06:04.890 --> 00:06:07.680
close to 0 and a little bit
more likely to get numbers

00:06:07.680 --> 00:06:08.200
further out.

00:06:08.200 --> 00:06:11.430
But it still is kind of
shifted or heavily weighted

00:06:11.430 --> 00:06:12.999
towards small numbers.

00:06:12.999 --> 00:06:14.790
And then if we had
another random variable,

00:06:14.790 --> 00:06:19.000
another chi-squared
distributed random variable--

00:06:19.000 --> 00:06:21.470
so then we have, let's say, Q3.

00:06:21.470 --> 00:06:25.760
And let's define
it as the sum of 3

00:06:25.760 --> 00:06:29.442
of these independent
variables, each of them

00:06:29.442 --> 00:06:31.150
that have a standard
normal distribution.

00:06:31.150 --> 00:06:36.600
So X1, X2 squared
plus X3 squared.

00:06:36.600 --> 00:06:39.990
Then all of a sudden, our
Q3-- this is Q2 right here--

00:06:39.990 --> 00:06:44.150
has a chi-squared distribution
with 3 degrees of freedom.

00:06:44.150 --> 00:06:45.922
And so this guy
right over here--

00:06:45.922 --> 00:06:47.130
that will be this green line.

00:06:47.130 --> 00:06:48.970
Maybe I should have
done this in green.

00:06:48.970 --> 00:06:51.204
This will be this
green line over here.

00:06:51.204 --> 00:06:52.620
And then notice,
now it's starting

00:06:52.620 --> 00:06:54.036
to become a little
bit more likely

00:06:54.036 --> 00:06:56.090
that you'd get values
in this range over here

00:06:56.090 --> 00:06:57.340
because you're taking the sum.

00:06:57.340 --> 00:06:59.130
Each of these are going
to be pretty small values,

00:06:59.130 --> 00:07:00.213
but you're taking the sum.

00:07:00.213 --> 00:07:02.627
So it starts to shift it a
little over to the right.

00:07:02.627 --> 00:07:04.460
And so the more degrees
of freedom you have,

00:07:04.460 --> 00:07:07.680
the further this lump
starts to move to the right

00:07:07.680 --> 00:07:09.850
and, to some degree, the
more symmetric it gets.

00:07:09.850 --> 00:07:11.332
And what's interesting
about this,

00:07:11.332 --> 00:07:13.790
I guess it's different than
almost every other distribution

00:07:13.790 --> 00:07:16.327
we've looked at, although
we've looked at others that

00:07:16.327 --> 00:07:18.410
have this property as well,
is that you can't have

00:07:18.410 --> 00:07:21.530
a value below 0 because
we're always just squaring

00:07:21.530 --> 00:07:22.160
these values.

00:07:22.160 --> 00:07:24.162
Each of these guys can
have values below 0.

00:07:24.162 --> 00:07:25.370
They're normally distributed.

00:07:25.370 --> 00:07:26.710
They could have negative values.

00:07:26.710 --> 00:07:29.090
But since we're squaring and
taking the sum of squares,

00:07:29.090 --> 00:07:30.810
this is always going
to be positive.

00:07:30.810 --> 00:07:32.770
And the place that this
is going to be useful--

00:07:32.770 --> 00:07:34.728
and we're going to see
in the next few videos--

00:07:34.728 --> 00:07:38.350
is in measuring essentially
error from an expected value.

00:07:38.350 --> 00:07:40.244
And if you took take
this total error,

00:07:40.244 --> 00:07:41.660
you can figure out
the probability

00:07:41.660 --> 00:07:48.682
of getting that total error
if you assume some parameters.

00:07:48.682 --> 00:07:50.640
And we'll talk more about
it in the next video.

00:07:50.640 --> 00:07:52.850
Now with that said, I
just want to show you

00:07:52.850 --> 00:07:56.620
how to read a chi-squared
distribution table.

00:07:56.620 --> 00:08:01.430
So if I were to ask you, if
this is our distribution--

00:08:01.430 --> 00:08:03.231
let me pick this
blue one right here.

00:08:03.231 --> 00:08:04.980
So over here, we have
2 degrees of freedom

00:08:04.980 --> 00:08:07.270
because we're adding 2
of these guys right here.

00:08:07.270 --> 00:08:15.880
If I were to ask you, what is
the probability of Q2 being

00:08:15.880 --> 00:08:21.860
greater than-- or, let
me put it this way.

00:08:21.860 --> 00:08:27.174
What is the probability of
Q2 being greater than 2.41?

00:08:27.174 --> 00:08:28.840
And I'm picking that
value for a reason.

00:08:28.840 --> 00:08:34.669
So I want the probability of
Q2 being greater than 2.41.

00:08:34.669 --> 00:08:39.250
What I want to do is I'll
look at a chi-square table

00:08:39.250 --> 00:08:40.090
like this.

00:08:40.090 --> 00:08:44.130
Q2 is a version of chi-squared
with 2 degrees of freedom.

00:08:44.130 --> 00:08:48.380
So I look at this row right
here under 2 degrees of freedom.

00:08:48.380 --> 00:08:52.990
And I want the probability of
getting a value above 2.41.

00:08:52.990 --> 00:08:56.350
And I picked 2.41 because
it's actually at this table.

00:08:56.350 --> 00:08:58.229
And so most of these
chi-squared-- the reason

00:08:58.229 --> 00:09:00.270
why we have these weird
numbers like this instead

00:09:00.270 --> 00:09:02.720
of whole numbers or
easy-to-read fractions

00:09:02.720 --> 00:09:04.740
is it is actually
driven by the p value.

00:09:04.740 --> 00:09:06.900
It's driven by the
probability of getting

00:09:06.900 --> 00:09:08.937
something larger
than that value.

00:09:08.937 --> 00:09:10.770
So normally you would
look at the other way.

00:09:10.770 --> 00:09:15.270
You'd say, OK, if I want to
say, what chi-squared value

00:09:15.270 --> 00:09:17.580
for 2 degrees of
freedom, there's

00:09:17.580 --> 00:09:20.270
a 30% chance of getting
something larger than that?

00:09:20.270 --> 00:09:21.642
Then I would look up 2.41.

00:09:21.642 --> 00:09:23.100
But I'm doing it
the other way just

00:09:23.100 --> 00:09:24.225
for the sake of this video.

00:09:24.225 --> 00:09:27.190
So if I want the probability
of this random variable

00:09:27.190 --> 00:09:32.470
right here being greater
than 2.41, or its p value,

00:09:32.470 --> 00:09:33.400
we read it right here.

00:09:33.400 --> 00:09:35.210
It is 30%.

00:09:35.210 --> 00:09:37.570
And just to visualize
it on this chart,

00:09:37.570 --> 00:09:40.060
this chi-square distribution--
this was Q2, the blue one,

00:09:40.060 --> 00:09:42.740
over here-- 2.41 is
going to sit-- let's see.

00:09:42.740 --> 00:09:44.080
This is 3.

00:09:44.080 --> 00:09:45.400
This is 2.5.

00:09:45.400 --> 00:09:49.060
So 2.41 is going to be
someplace right around here.

00:09:49.060 --> 00:09:50.530
So essentially,
what that table is

00:09:50.530 --> 00:09:58.190
telling us is, this entire
area under this blue line right

00:09:58.190 --> 00:09:59.420
here, what is that?

00:09:59.420 --> 00:10:03.560
And that right there is
going to be 30% of-- well,

00:10:03.560 --> 00:10:07.050
it's going to be 0.3.

00:10:07.050 --> 00:10:09.846
Or you could view it as
30% of the entire area

00:10:09.846 --> 00:10:12.220
under this curve, because
obviously all the probabilities

00:10:12.220 --> 00:10:13.600
have to add up to 1.

00:10:13.600 --> 00:10:15.725
So that's our intro to the
chi-square distribution.

00:10:15.725 --> 00:10:17.100
In the next video,
we're actually

00:10:17.100 --> 00:10:21.724
going to use it to make some,
or to test some, inferences.


Covariance and the regression line

WEBVTT
Kind: captions
Language: en

00:00:00.610 --> 00:00:02.780
What I want to do in this
video is introduce you

00:00:02.780 --> 00:00:09.160
to the idea of the covariance
between two random variables.

00:00:09.160 --> 00:00:16.129
And it's defined as the
expected value of the distance--

00:00:16.129 --> 00:00:17.760
or I guess the product
of the distances

00:00:17.760 --> 00:00:21.350
of each random variable
from their mean,

00:00:21.350 --> 00:00:23.209
or from their expected value.

00:00:23.209 --> 00:00:24.500
So let me just write that down.

00:00:24.500 --> 00:00:28.280
So I'll have X first, I'll
do this in another color.

00:00:28.280 --> 00:00:30.660
So it's the expected
value of random variable X

00:00:30.660 --> 00:00:34.730
minus the expected value
of X. You could view this

00:00:34.730 --> 00:00:38.030
as the population
mean of X times--

00:00:38.030 --> 00:00:40.220
and then this is a
random variable y--

00:00:40.220 --> 00:00:45.880
so times the distance from
Y to its expected value

00:00:45.880 --> 00:00:53.350
or the population mean of y.

00:00:53.350 --> 00:00:55.670
And if it doesn't make
a lot of intuitive sense

00:00:55.670 --> 00:00:57.422
yet-- well, one, you
could just always

00:00:57.422 --> 00:00:59.380
kind of think about what
it's doing play around

00:00:59.380 --> 00:01:01.150
with some numbers here.

00:01:01.150 --> 00:01:04.360
But the reality is it's saying
how much they vary together.

00:01:04.360 --> 00:01:08.936
So you always take an X and a
y for each of the data points.

00:01:08.936 --> 00:01:10.560
Let's say you had
the whole population.

00:01:10.560 --> 00:01:13.370
So every X and Y that
kind of go together

00:01:13.370 --> 00:01:16.630
with each other that are
coordinate you put into this.

00:01:16.630 --> 00:01:20.230
And what happens is-- let's
say that X is above its mean

00:01:20.230 --> 00:01:22.220
when Y is below its mean.

00:01:22.220 --> 00:01:25.290
So let's say that in the
population you had the point.

00:01:25.290 --> 00:01:28.760
So one instantiation
of the random variables

00:01:28.760 --> 00:01:30.900
you sample once
from the universe

00:01:30.900 --> 00:01:38.840
and you get X is equal to 1 and
that Y is equal to-- let's say

00:01:38.840 --> 00:01:40.390
Y is equal to 3.

00:01:40.390 --> 00:01:43.550
And let's say that you
knew ahead of time,

00:01:43.550 --> 00:01:47.070
that the expected
value of X is 0.

00:01:47.070 --> 00:01:53.200
And let's say that the expected
value of Y is equal to 4.

00:01:53.200 --> 00:01:55.990
So in this situation,
what just happened?

00:01:55.990 --> 00:01:57.870
Now we don't know the
entire covariance,

00:01:57.870 --> 00:02:02.000
we only have one sample here
of this random variable.

00:02:02.000 --> 00:02:03.490
But what just happened here?

00:02:03.490 --> 00:02:06.150
We have one minus-- so we're
just going to calculate,

00:02:06.150 --> 00:02:07.970
we're not going to calculate
the entire expected value,

00:02:07.970 --> 00:02:09.511
I just want to
calculate what happens

00:02:09.511 --> 00:02:12.020
when we do what's inside
the expected value.

00:02:12.020 --> 00:02:18.400
We'll have 1 minus 0, so you'll
have a 1 times a 3 minus 4,

00:02:18.400 --> 00:02:20.419
times a negative 1.

00:02:20.419 --> 00:02:22.460
So you're going to have
1 times negative 1, which

00:02:22.460 --> 00:02:23.980
is negative 1.

00:02:23.980 --> 00:02:25.430
And what is that telling us?

00:02:25.430 --> 00:02:28.530
Well, it's telling us at least
for this sample, this one time

00:02:28.530 --> 00:02:31.382
that we sampled the
random variables X and Y,

00:02:31.382 --> 00:02:35.740
X was above it's
expected value when

00:02:35.740 --> 00:02:38.460
Y was below its expected value.

00:02:38.460 --> 00:02:41.130
And if we kept doing this, let's
say for the entire population

00:02:41.130 --> 00:02:42.960
this happened, then
it would make sense

00:02:42.960 --> 00:02:45.560
that they have a
negative covariance.

00:02:45.560 --> 00:02:47.770
When one goes up, the
other one goes down.

00:02:47.770 --> 00:02:49.990
When one goes down,
the other one goes up.

00:02:49.990 --> 00:02:51.750
If they both go
up together, they

00:02:51.750 --> 00:02:54.350
would have a positive variance
or they both go down together.

00:02:54.350 --> 00:02:56.760
And the degree to which
they do it together

00:02:56.760 --> 00:02:59.020
will tell you the magnitude
of the covariance.

00:02:59.020 --> 00:03:01.190
Hopefully that gives you
a little bit of intuition

00:03:01.190 --> 00:03:03.380
about what the covariance
is trying to tell us.

00:03:03.380 --> 00:03:05.880
But the more important thing
that I want to do in this video

00:03:05.880 --> 00:03:08.200
is to connect this formula.

00:03:08.200 --> 00:03:11.119
I want to connect to this
definition of covariance

00:03:11.119 --> 00:03:13.660
to everything we've been doing
with least squared regression.

00:03:13.660 --> 00:03:15.550
And really it's just
kind of a fun math thing

00:03:15.550 --> 00:03:19.420
to do to show you all
of these connections,

00:03:19.420 --> 00:03:21.800
and where, really, the
definition of covariance

00:03:21.800 --> 00:03:23.130
really becomes useful.

00:03:23.130 --> 00:03:25.450
And I really do think it's
motivated to a large degree

00:03:25.450 --> 00:03:27.994
by where it shows
up in regressions.

00:03:27.994 --> 00:03:29.660
And this is all stuff
that we've kind of

00:03:29.660 --> 00:03:32.430
seen before, you're just going
to see it in a different way.

00:03:32.430 --> 00:03:36.790
So this whole video,
I'm just going

00:03:36.790 --> 00:03:40.280
to rewrite this definition of
covariance right over here.

00:03:40.280 --> 00:03:45.730
So this is going to be the
same thing as the expected

00:03:45.730 --> 00:03:50.070
value of-- and I'm
just going to multiply

00:03:50.070 --> 00:03:51.740
these two binomials in here.

00:03:51.740 --> 00:03:55.050
So the expected value
of our random variable X

00:03:55.050 --> 00:04:01.280
times our random variable
Y minus-- well, I'll

00:04:01.280 --> 00:04:02.240
just do the X first.

00:04:02.240 --> 00:04:06.050
So plus X times the negative
expected value of Y.

00:04:06.050 --> 00:04:18.543
So I'll just say minus X
times the expected value of Y.

00:04:18.543 --> 00:04:20.709
And that negative sign comes
from this negative sign

00:04:20.709 --> 00:04:21.910
right over here.

00:04:21.910 --> 00:04:31.220
And then we have minus
expected value of X times Y,

00:04:31.220 --> 00:04:33.404
just doing the distributive
property twice,

00:04:33.404 --> 00:04:35.820
and then finally you have the
negative expected value of X

00:04:35.820 --> 00:04:37.940
times a negative
expected value of Y.

00:04:37.940 --> 00:04:39.630
And the negatives cancel out.

00:04:39.630 --> 00:04:45.360
And so you're just going to have
plus the expected value of X

00:04:45.360 --> 00:04:53.320
times the expected value
of Y. And of course, it's

00:04:53.320 --> 00:04:55.025
the expected value
of this entire thing.

00:04:59.569 --> 00:05:01.110
Now let's see if we
can rewrite this.

00:05:01.110 --> 00:05:02.900
Well the expected
value of the sum

00:05:02.900 --> 00:05:06.280
of a bunch of random variables,
or the sum and difference

00:05:06.280 --> 00:05:09.120
of a bunch of random variables,
is just the sum or difference

00:05:09.120 --> 00:05:10.340
of their expected value.

00:05:10.340 --> 00:05:13.250
So this is going to
be the same thing.

00:05:13.250 --> 00:05:15.560
And remember, expected
value, in a lot of contexts,

00:05:15.560 --> 00:05:17.550
you could view it as
just the arithmetic mean.

00:05:17.550 --> 00:05:19.580
Or, in a continuous
distribution,

00:05:19.580 --> 00:05:21.570
you could view it as a
probability weighted sum

00:05:21.570 --> 00:05:24.450
or probability weighted
integral, either way.

00:05:24.450 --> 00:05:26.410
We've seen it before, I think.

00:05:26.410 --> 00:05:27.460
So let's rewrite this.

00:05:27.460 --> 00:05:31.330
So this is equal to the expected
value of the random variables

00:05:31.330 --> 00:05:38.570
X and Y. X times Y. Trying to
keep them color-coded for you.

00:05:38.570 --> 00:05:42.550
And then we have minus X
times the expected value of Y.

00:05:42.550 --> 00:05:49.860
So then we're going to have
minus the expected value of X

00:05:49.860 --> 00:06:03.530
times the expected value of
Y. Stay with the right colors.

00:06:03.530 --> 00:06:06.170
Then you're going to have
minus the expected value

00:06:06.170 --> 00:06:11.490
of this thing-- I'll close the
parentheses-- of this thing

00:06:11.490 --> 00:06:12.250
right over here.

00:06:12.250 --> 00:06:17.942
Expected value of
X times Y. I know

00:06:17.942 --> 00:06:19.650
this might look really
confusing with all

00:06:19.650 --> 00:06:21.334
the embedded expected values.

00:06:21.334 --> 00:06:23.000
But one way to think
about is the things

00:06:23.000 --> 00:06:24.250
that already have
the expected values,

00:06:24.250 --> 00:06:25.710
you can view these as numbers.

00:06:25.710 --> 00:06:27.169
You've already used
them as knowns.

00:06:27.169 --> 00:06:29.668
We're actually going to take
them out of the expected value,

00:06:29.668 --> 00:06:31.650
because the expected
value of an expected value

00:06:31.650 --> 00:06:33.080
is the same thing as
the expected value.

00:06:33.080 --> 00:06:35.579
Actually let me write this over
here, just remind ourselves.

00:06:35.579 --> 00:06:41.730
The expected value
of X is just going

00:06:41.730 --> 00:06:48.049
to be the expected value
of X. Think of it this way.

00:06:48.049 --> 00:06:49.590
You could view this
as the population

00:06:49.590 --> 00:06:51.290
mean for the random variable.

00:06:51.290 --> 00:06:53.656
So that's just going to be
a known, it's out there,

00:06:53.656 --> 00:06:54.530
it's in the universe.

00:06:54.530 --> 00:06:59.120
So the expected value of that
is just going to be itself.

00:06:59.120 --> 00:07:03.740
If the population mean, or
the expected value of X is 5--

00:07:03.740 --> 00:07:05.989
this is like saying the
expected value of 5.

00:07:05.989 --> 00:07:07.530
Well the expected
value of 5 is going

00:07:07.530 --> 00:07:10.130
to be 5, which is the same thing
as the expected value of X.

00:07:10.130 --> 00:07:11.160
Hopefully that
makes sense, we're

00:07:11.160 --> 00:07:12.425
going to use that in a second.

00:07:12.425 --> 00:07:13.300
So we're almost done.

00:07:13.300 --> 00:07:16.060
We did the expected value of
this and we have one term left.

00:07:16.060 --> 00:07:19.296
And then the final term, the
expected value of this guy.

00:07:19.296 --> 00:07:21.920
And here, we can actually use a
property right from the get go.

00:07:21.920 --> 00:07:22.950
I'll write it down.

00:07:22.950 --> 00:07:26.630
So the expected value of--
get some big brackets up--

00:07:26.630 --> 00:07:28.660
of this thing right over here.

00:07:28.660 --> 00:07:37.790
Expected value of X times the
expected value of Y. And let's

00:07:37.790 --> 00:07:39.534
see if we can simplify
it right here.

00:07:39.534 --> 00:07:41.450
So this is just going
to be the expected value

00:07:41.450 --> 00:07:44.350
of the product of these
two random variables.

00:07:44.350 --> 00:07:46.042
I'll just leave
that the way it is.

00:07:46.042 --> 00:07:47.500
So let me just--
the stuff that I'm

00:07:47.500 --> 00:07:50.000
going to leave the way it is
I'm just going to freeze them.

00:07:50.000 --> 00:07:54.850
So the expected value of XY.

00:07:54.850 --> 00:07:56.500
Now what do we have over here?

00:07:56.500 --> 00:07:59.322
We have the expected
value of X times--

00:07:59.322 --> 00:08:00.780
once again, you
can kind of view it

00:08:00.780 --> 00:08:02.321
if you go back to
what we just said--

00:08:02.321 --> 00:08:05.160
is this is just going to be a
number, expected value of Y,

00:08:05.160 --> 00:08:06.440
so we can just bring this out.

00:08:06.440 --> 00:08:08.352
If this was the
expected value of 3X,

00:08:08.352 --> 00:08:10.810
would be the same thing as 3
times the expected value of X.

00:08:10.810 --> 00:08:18.420
So we can rewrite this as
negative expected value of Y

00:08:18.420 --> 00:08:25.994
times the expected value of
X. You can kind of view this

00:08:25.994 --> 00:08:27.660
as we took it out of
the expected value,

00:08:27.660 --> 00:08:29.630
we factored it out.

00:08:29.630 --> 00:08:31.170
So just like that.

00:08:31.170 --> 00:08:33.710
And then you have minus.

00:08:33.710 --> 00:08:34.880
Same thing over here.

00:08:34.880 --> 00:08:36.559
You can factor out
this expected value

00:08:36.559 --> 00:08:42.530
of X. Minus the expected
value of X times the expected

00:08:42.530 --> 00:08:53.750
value of Y. This is getting
confusing with all the E's

00:08:53.750 --> 00:08:55.060
laying around.

00:08:55.060 --> 00:08:57.250
And then finally,
the expected value

00:08:57.250 --> 00:08:59.200
of this thing, of
two expected values,

00:08:59.200 --> 00:09:00.670
well that's just going to
be the product of those two

00:09:00.670 --> 00:09:01.537
expected values.

00:09:01.537 --> 00:09:03.120
So that's just going
to be plus-- I'll

00:09:03.120 --> 00:09:11.180
freeze this-- expected
value of X times

00:09:11.180 --> 00:09:15.010
the expected value of Y.

00:09:15.010 --> 00:09:16.150
Now what do we have here?

00:09:16.150 --> 00:09:20.180
We have expected value of Y
times the expected value of X.

00:09:20.180 --> 00:09:23.010
And then we are subtracting
the expected value

00:09:23.010 --> 00:09:25.660
of X times the
expected value of Y.

00:09:25.660 --> 00:09:28.340
These two things are
the exact same thing.

00:09:28.340 --> 00:09:28.840
Right?

00:09:28.840 --> 00:09:30.590
So this is going to be--
and actually look at this.

00:09:30.590 --> 00:09:33.080
We're subtracting it twice
and then we have one more.

00:09:33.080 --> 00:09:35.180
These are all the same thing.

00:09:35.180 --> 00:09:38.240
This is the expected value of Y
times the expected value of X.

00:09:38.240 --> 00:09:41.024
This is the expected value of Y
times the expected value of X,

00:09:41.024 --> 00:09:42.440
just written in a
different order.

00:09:42.440 --> 00:09:46.200
And this is the expected value
of Y times the expected value

00:09:46.200 --> 00:09:49.650
of X. We're subtracting it
twice and then we're adding it.

00:09:49.650 --> 00:09:53.100
Or, one way to think about it
is that this guy and that guy

00:09:53.100 --> 00:09:53.960
will cancel out.

00:09:53.960 --> 00:09:56.001
You could have also picked
that guy and that guy.

00:09:56.001 --> 00:09:57.230
But what do we have left?

00:09:57.230 --> 00:09:59.650
We have the covariance of
these two random variables.

00:09:59.650 --> 00:10:02.465
X and Y are equal to the
expected value of-- I'll

00:10:02.465 --> 00:10:04.215
switch back to my
colors just because this

00:10:04.215 --> 00:10:09.220
is the final result-- the
expected value of X times

00:10:09.220 --> 00:10:22.869
the expected value of
the product of XY minus--

00:10:22.869 --> 00:10:23.410
what is this?

00:10:23.410 --> 00:10:34.220
The expected value of Y times
the expected value of X.

00:10:34.220 --> 00:10:37.430
Now you can calculate
these expected values

00:10:37.430 --> 00:10:40.080
if you know everything about
the probability distribution

00:10:40.080 --> 00:10:42.480
or density functions for each
of these random variables.

00:10:42.480 --> 00:10:44.122
Or if you had the
entire population

00:10:44.122 --> 00:10:45.580
that you're sampling
from, whenever

00:10:45.580 --> 00:10:47.912
you take an instantiation
of these random variables.

00:10:47.912 --> 00:10:49.370
But let's say you
just had a sample

00:10:49.370 --> 00:10:50.453
of these random variables.

00:10:50.453 --> 00:10:54.100
How could you estimate them?

00:10:54.100 --> 00:10:58.816
Well, if you were estimating
it, the expected value,

00:10:58.816 --> 00:11:00.940
and let's say you just have
a bunch of data points,

00:11:00.940 --> 00:11:01.690
a bunch of coordinates.

00:11:01.690 --> 00:11:03.670
And I think you'll start
to see how this relates

00:11:03.670 --> 00:11:04.919
to what we do with regression.

00:11:07.560 --> 00:11:11.360
The expected value
of X times Y, it

00:11:11.360 --> 00:11:14.130
can be approximated
by the sample mean

00:11:14.130 --> 00:11:16.690
of the products of
X and Y. This is

00:11:16.690 --> 00:11:19.230
going to be the sample
mean of X and Y.

00:11:19.230 --> 00:11:21.920
You take each of
your XY associations,

00:11:21.920 --> 00:11:24.660
take their product, and then
take the mean of all of them.

00:11:24.660 --> 00:11:26.535
So that's going to be
the product of X and Y.

00:11:26.535 --> 00:11:28.190
And then this thing
right over here,

00:11:28.190 --> 00:11:33.030
the expected value of Y that can
be approximated by the sample

00:11:33.030 --> 00:11:35.610
mean of Y, and the
expected value of X

00:11:35.610 --> 00:11:38.480
can be approximated by
the sample mean of X.

00:11:38.480 --> 00:11:42.370
So what can the covariance
of two random variables

00:11:42.370 --> 00:11:44.060
be approximated by?

00:11:44.060 --> 00:11:45.680
What can it be approximated by?

00:11:45.680 --> 00:11:52.480
Well this right here is the
mean of their product from

00:11:52.480 --> 00:12:01.110
your sample minus the mean of
your sample Y's times the mean

00:12:01.110 --> 00:12:02.075
of your sample X's.

00:12:04.880 --> 00:12:06.940
And this should start
looking familiar.

00:12:06.940 --> 00:12:10.500
This should look a little bit
familiar, because what is this?

00:12:10.500 --> 00:12:12.540
This was the numerator.

00:12:12.540 --> 00:12:16.880
This right here is
the numerator when

00:12:16.880 --> 00:12:19.874
we were trying to figure out the
slope of the regression line.

00:12:19.874 --> 00:12:22.290
So when we tried to figure out
the slope of the regression

00:12:22.290 --> 00:12:25.040
line, we had the-- let me just
rewrite the formula here just

00:12:25.040 --> 00:12:30.560
to remind you-- it was literally
the mean of the products

00:12:30.560 --> 00:12:35.530
of each of our data points,
or the XY's, minus the mean

00:12:35.530 --> 00:12:38.370
of Y's times the
mean of the X's.

00:12:38.370 --> 00:12:43.200
All of that over the
mean the X squareds.

00:12:43.200 --> 00:12:45.700
And you could even
view it as this,

00:12:45.700 --> 00:12:48.570
over the mean of
the X times the X's.

00:12:48.570 --> 00:12:52.500
But I could just write
the X squareds, over here,

00:12:52.500 --> 00:12:59.490
minus the mean of X squared.

00:12:59.490 --> 00:13:02.774
This is how we figured out the
slope of our regression line.

00:13:02.774 --> 00:13:04.440
Or maybe a better way
to think about it,

00:13:04.440 --> 00:13:06.180
if we assume in
our regression line

00:13:06.180 --> 00:13:08.675
that the points
that we have were

00:13:08.675 --> 00:13:12.410
a sample from an entire
universe of possible points,

00:13:12.410 --> 00:13:15.265
then you could say that
we are approximating

00:13:15.265 --> 00:13:16.640
the slope of our
aggression line.

00:13:16.640 --> 00:13:19.550
And you might see this little
hat notation in a lot of books.

00:13:19.550 --> 00:13:20.883
I don't want you to be confused.

00:13:20.883 --> 00:13:23.640
They are saying that you're
approximating the population's

00:13:23.640 --> 00:13:27.340
regression line
from a sample of it.

00:13:27.340 --> 00:13:29.300
Now, this right
here-- so everything

00:13:29.300 --> 00:13:31.990
we've learned right now-- this
right here is the covariance,

00:13:31.990 --> 00:13:37.120
or this is an estimate of
the covariance of X and Y.

00:13:37.120 --> 00:13:38.620
Now what is this over here?

00:13:38.620 --> 00:13:40.170
Well, I just said,
you could rewrite

00:13:40.170 --> 00:13:43.040
this very easily
as-- this bottom part

00:13:43.040 --> 00:13:47.310
right here-- you could write as
the mean of X times X-- that's

00:13:47.310 --> 00:13:51.810
the same thing as X
squared-- minus the mean of X

00:13:51.810 --> 00:13:53.660
times the mean of X, right?

00:13:53.660 --> 00:13:55.440
That's what the mean
of X squared is.

00:13:55.440 --> 00:13:56.580
Well, what's this?

00:13:56.580 --> 00:14:03.060
Well, you could view this as
the covariance of X with X.

00:14:03.060 --> 00:14:05.230
But we've actually
already seen this.

00:14:05.230 --> 00:14:07.370
And I've actually
shown you many,

00:14:07.370 --> 00:14:09.870
many videos ago when we first
learned about it what this is.

00:14:09.870 --> 00:14:12.640
The covariance of a random
variable with itself

00:14:12.640 --> 00:14:18.680
is really just the variance
of that random variable.

00:14:18.680 --> 00:14:20.470
And you could verify
it for yourself.

00:14:20.470 --> 00:14:23.990
If you change this
Y to an X, this

00:14:23.990 --> 00:14:25.870
becomes X minus
the expected value

00:14:25.870 --> 00:14:28.590
of X times X minus
expected value of X.

00:14:28.590 --> 00:14:30.720
Or that's the
expected value of X

00:14:30.720 --> 00:14:32.940
minus the expected
value of X squared.

00:14:32.940 --> 00:14:35.270
That's your definition
of variance.

00:14:35.270 --> 00:14:39.650
So another way of thinking about
the slope of our aggression

00:14:39.650 --> 00:14:45.160
line, it can be literally viewed
as the covariance of our two

00:14:45.160 --> 00:14:53.350
random variables over
the variance of X.

00:14:53.350 --> 00:14:55.650
Or you can kind of view it
as the independent random

00:14:55.650 --> 00:14:56.450
variable.

00:14:56.450 --> 00:14:59.415
That right there is the
slope of our regression line.

00:14:59.415 --> 00:15:01.040
Anyway, I thought
that was interesting.

00:15:01.040 --> 00:15:02.940
And I wanted to make
connections between things

00:15:02.940 --> 00:15:04.648
you see in different
parts of statistics,

00:15:04.648 --> 00:15:07.430
and show you that they
really are connected.


Calculating R-squared

WEBVTT
Kind: captions
Language: en

00:00:00.510 --> 00:00:02.730
In the last video, we were able
to find the equation for

00:00:02.730 --> 00:00:07.480
the regression line for these
four data points.

00:00:07.480 --> 00:00:09.460
What I want to do in this video
is figure out the r

00:00:09.460 --> 00:00:10.760
squared for these data points.

00:00:10.760 --> 00:00:13.670
Figure out how good this
line fits the data.

00:00:13.670 --> 00:00:16.040
Or even better, figure out the
percentage-- which is really

00:00:16.040 --> 00:00:19.640
the same thing-- of the
variation of these data

00:00:19.640 --> 00:00:23.220
points, especially the variation
in y, that is due

00:00:23.220 --> 00:00:26.760
to, or that can be explained
by variation in x.

00:00:26.760 --> 00:00:27.911
And to do that, I'm actually
going to get

00:00:27.911 --> 00:00:29.010
a spreadsheet out.

00:00:29.010 --> 00:00:31.120
I've actually tried to do this
with a calculator and it's

00:00:31.120 --> 00:00:31.830
much harder.

00:00:31.830 --> 00:00:34.710
So hopefully this doesn't
confuse you too much to use a

00:00:34.710 --> 00:00:35.280
spreadsheet.

00:00:35.280 --> 00:00:36.650
And I'm a make a couple
of columns here.

00:00:36.650 --> 00:00:38.440
And spreadsheets actually have
functions that'll do all of

00:00:38.440 --> 00:00:40.280
this automatically, but I really
want to do it so that

00:00:40.280 --> 00:00:42.360
you could do it by hand
if you had to.

00:00:42.360 --> 00:00:44.040
So I'm going to make a couple
of columns here.

00:00:44.040 --> 00:00:45.780
This is going to
be my x column.

00:00:45.780 --> 00:00:48.360
This is going to
be my y column.

00:00:48.360 --> 00:00:50.550
This is going to be the column--
I'll call this y

00:00:50.550 --> 00:00:56.220
star-- this'll be the y value
that our line predicts based

00:00:56.220 --> 00:00:57.930
on our x value.

00:00:57.930 --> 00:01:08.930
This is going to be the
error with the line.

00:01:08.930 --> 00:01:10.750
Let me caught it the squared
error with the line.

00:01:20.070 --> 00:01:21.510
I don't want us to take
up too much space.

00:01:28.280 --> 00:01:41.760
And then the next one, I'm
going to have the squared

00:01:41.760 --> 00:01:56.040
variation for that y value
from the mean y.

00:01:56.040 --> 00:01:59.940
And I think these columns by
themselves will be enough for

00:01:59.940 --> 00:02:00.680
us to do everything.

00:02:00.680 --> 00:02:02.220
So let's first put all
the data points in.

00:02:02.220 --> 00:02:05.590
So we had negative 2
comma negative 3.

00:02:05.590 --> 00:02:06.620
That was one data point.

00:02:06.620 --> 00:02:08.740
Negative 1 comma negative 1.

00:02:08.740 --> 00:02:10.889
And we had 1 comma 2.

00:02:10.889 --> 00:02:14.010
Then we have 4 comma 3.

00:02:14.010 --> 00:02:19.145
Now, what does our
line predict?

00:02:19.145 --> 00:02:21.360
Well our line says, you give
me an x value, I'm going to

00:02:21.360 --> 00:02:23.530
tell you what y value
I'll predict.

00:02:23.530 --> 00:02:27.920
So when x is equal to negative
2, the y value on the line is

00:02:27.920 --> 00:02:29.230
going to be the slope.

00:02:29.230 --> 00:02:34.060
So this is going to be equal
to 41 divided by

00:02:34.060 --> 00:02:38.910
42 times our x value.

00:02:38.910 --> 00:02:40.600
And I just selected that cell.

00:02:40.600 --> 00:02:43.470
And just a little bit of a
primer on spreadsheets, I'm

00:02:43.470 --> 00:02:46.020
selecting the cell D2.

00:02:46.020 --> 00:02:48.680
I was able to just move my
cursor over and select that.

00:02:48.680 --> 00:02:51.200
But that tells me the x value.

00:02:51.200 --> 00:02:53.670
Minus 5/21.

00:02:53.670 --> 00:02:59.110
Minus 5 divided by 21.

00:02:59.110 --> 00:03:02.080
Just like that.

00:03:02.080 --> 00:03:04.800
So just to be clear of what
we're even doing.

00:03:04.800 --> 00:03:07.650
This y star here, I
got negative 2.19.

00:03:07.650 --> 00:03:09.260
That tells us at this
point right over

00:03:09.260 --> 00:03:17.170
here is negative 2.19.

00:03:17.170 --> 00:03:18.476
So when we figure out the error,
we're going to figure

00:03:18.476 --> 00:03:25.600
out the distance between
negative 3, that's our y

00:03:25.600 --> 00:03:28.860
value, and negative 2.19.

00:03:28.860 --> 00:03:31.430
So let's do that.

00:03:31.430 --> 00:03:35.970
So the error is just going to
be equal to our y value.

00:03:35.970 --> 00:03:37.220
That's cell E2.

00:03:39.660 --> 00:03:45.370
Minus the value that our
line would predict.

00:03:45.370 --> 00:03:47.520
So just that value is
the actual error.

00:03:47.520 --> 00:03:48.770
But we want to square it.

00:03:56.870 --> 00:03:58.090
And then, the next thing
we want to do

00:03:58.090 --> 00:04:00.480
is the squared distance.

00:04:00.480 --> 00:04:03.730
so this is equal to the squared
distance of our y

00:04:03.730 --> 00:04:05.315
value from the y's mean.

00:04:05.315 --> 00:04:07.680
So what's the mean of the y's?

00:04:07.680 --> 00:04:09.170
Mean of the y's is 1/4.

00:04:09.170 --> 00:04:13.290
So minus 0.25, is the
same thing is 1/4.

00:04:13.290 --> 00:04:17.260
And we also want
to square that.

00:04:17.260 --> 00:04:19.720
Now, this is what's fun
about spreadsheets.

00:04:19.720 --> 00:04:23.150
I can apply those formulas
to every row now.

00:04:23.150 --> 00:04:25.330
And notice, what it did
when I did that.

00:04:25.330 --> 00:04:29.430
Now all of a sudden, this is the
y value that my line would

00:04:29.430 --> 00:04:31.590
predict, it's now using
this x value and

00:04:31.590 --> 00:04:33.230
sticking it over here.

00:04:33.230 --> 00:04:36.120
It's now figuring out the square
distance from the line

00:04:36.120 --> 00:04:40.170
using what the line would
predict and using the

00:04:40.170 --> 00:04:41.630
y value, this one.

00:04:41.630 --> 00:04:45.010
And then does the same
thing over here.

00:04:45.010 --> 00:04:47.090
It's figures out the squared
distance of this y

00:04:47.090 --> 00:04:51.060
value from the mean.

00:04:51.060 --> 00:04:53.650
So what is the total squared
error with the line?

00:04:53.650 --> 00:04:55.620
So let me just sum this up.

00:04:55.620 --> 00:04:59.040
The total squared error
with the line is 2.73.

00:04:59.040 --> 00:05:05.920
And then the total variation
from the mean, squared

00:05:05.920 --> 00:05:10.560
distances from the mean
of the y, are 22.75.

00:05:10.560 --> 00:05:14.790
So let me be very clear
what this is.

00:05:14.790 --> 00:05:18.350
So let me write these
numbers down.

00:05:18.350 --> 00:05:20.230
I'll write it up here so we
can keep looking at this

00:05:20.230 --> 00:05:22.890
actual graph.

00:05:22.890 --> 00:05:27.400
So are squared error versus our
line, our total squared

00:05:27.400 --> 00:05:31.600
error, we just computed
to be 2.74.

00:05:31.600 --> 00:05:32.860
I rounded a little bit.

00:05:32.860 --> 00:05:35.240
And what that is, is you take
each of these data points'

00:05:35.240 --> 00:05:37.500
vertical distance to the line.

00:05:37.500 --> 00:05:41.050
So this distance squared, plus
this distance squared, plus

00:05:41.050 --> 00:05:43.740
this distance squared, plus
this distance squared.

00:05:43.740 --> 00:05:46.030
That's all we just calculated
on Excel.

00:05:46.030 --> 00:05:52.450
And that total squared variation
to the line is 2.74.

00:05:52.450 --> 00:05:54.680
Or total squared error
with the line.

00:05:54.680 --> 00:05:57.380
And then the other number we
figured out was the total

00:05:57.380 --> 00:05:58.190
distance from the mean.

00:05:58.190 --> 00:06:00.780
So the mean here is
y is equal to 1/4.

00:06:00.780 --> 00:06:03.980
So that's going to be
right over here.

00:06:08.510 --> 00:06:09.590
This is 1/2.

00:06:09.590 --> 00:06:10.840
So right over here.

00:06:14.505 --> 00:06:17.070
So this is our mean y value.

00:06:19.970 --> 00:06:22.630
Or the central tendency
for our y values.

00:06:22.630 --> 00:06:26.590
And so what we calculated next
was the total error, the

00:06:26.590 --> 00:06:30.730
squared error, from the
means of our y values.

00:06:30.730 --> 00:06:34.850
That's what we calculated over
here in the spreadsheet.

00:06:34.850 --> 00:06:36.320
You see in the formula.

00:06:36.320 --> 00:06:41.690
It is this number, E2, minus
0.25, which is the mean of our

00:06:41.690 --> 00:06:43.150
y's squared.

00:06:43.150 --> 00:06:44.300
That's exactly what
we calculated.

00:06:44.300 --> 00:06:46.170
We calculated for each
of the y values.

00:06:46.170 --> 00:06:47.210
And then we summed
them all up.

00:06:47.210 --> 00:06:49.820
It's 22.75.

00:06:49.820 --> 00:06:58.470
It is equal to 22.75.

00:06:58.470 --> 00:07:01.600
So this is essentially
the error that the

00:07:01.600 --> 00:07:03.570
line does not explain.

00:07:03.570 --> 00:07:06.060
This is the total error,
this is the total

00:07:06.060 --> 00:07:07.330
variation of the numbers.

00:07:07.330 --> 00:07:09.760
So if you wanted to know the
percentage of the total

00:07:09.760 --> 00:07:13.700
variation that is not explained
by the line, you

00:07:13.700 --> 00:07:15.680
could take this number divided
by this number.

00:07:15.680 --> 00:07:23.090
So 2.74 over 22.75.

00:07:23.090 --> 00:07:39.440
This tells us the percentage
of total variation not

00:07:39.440 --> 00:07:43.050
explained by the line or
by the variation in x.

00:07:50.210 --> 00:07:53.000
And so what is this number
going to be?

00:07:53.000 --> 00:07:56.590
I can just use Excel for this.

00:07:56.590 --> 00:08:02.500
So I'm just going to divide this
number divided by this

00:08:02.500 --> 00:08:04.290
number right over there.

00:08:04.290 --> 00:08:07.430
I get 0.12.

00:08:07.430 --> 00:08:16.000
So this is equal to 0.12.

00:08:16.000 --> 00:08:19.300
Or another way to think about
it is 12% of the total

00:08:19.300 --> 00:08:22.870
variation is not explained
by the variation in x.

00:08:22.870 --> 00:08:26.510
The total squared distance
between each of the points or

00:08:26.510 --> 00:08:29.260
their kind of spread, their
variation, is not explain by

00:08:29.260 --> 00:08:30.170
the variation in x.

00:08:30.170 --> 00:08:32.240
So if you want the amount that
is explained by the variance

00:08:32.240 --> 00:08:34.579
in x, you just subtract
that from 1.

00:08:34.579 --> 00:08:36.490
So let me write it
right over here.

00:08:36.490 --> 00:08:40.669
So we have our r squared, which
is the percent of the

00:08:40.669 --> 00:08:42.820
total variation that is
explained by x, is going to be

00:08:42.820 --> 00:08:46.350
1 the minus that 0.12 that
we just calculated.

00:08:51.540 --> 00:08:55.610
Which is going to be 0.88.

00:08:55.610 --> 00:08:58.180
So our r squared here is 0.88.

00:08:58.180 --> 00:08:59.630
It's very, very close to 1.

00:08:59.630 --> 00:09:02.170
The highest number
it can be is 1.

00:09:02.170 --> 00:09:05.610
So what this tells us, or a way
to interpret this, is that

00:09:05.610 --> 00:09:18.210
88% of the total variation of
these y values is explained by

00:09:18.210 --> 00:09:20.610
the line or by the
variation in x.

00:09:23.230 --> 00:09:26.040
And you can see that it looks
like a pretty good fit.

00:09:26.040 --> 00:09:27.610
Each of these aren't too far.

00:09:31.970 --> 00:09:33.790
Each of these points are
definitely much closer to the

00:09:33.790 --> 00:09:35.510
line than they are
to the mean line.

00:09:38.010 --> 00:09:41.830
In fact, all of them are closer
to our actual line than

00:09:41.830 --> 00:09:43.630
to the mean.


Proof (part 4) minimizing squared error to regression line

WEBVTT
Kind: captions
Language: en

00:00:00.620 --> 00:00:03.690
So if you've gotten this far,
you've been waiting for

00:00:03.690 --> 00:00:08.730
several videos to get to the
optimal line that minimizes

00:00:08.730 --> 00:00:10.500
the squared distance to
all of those points.

00:00:10.500 --> 00:00:11.930
So let's just get to
the punch line.

00:00:11.930 --> 00:00:14.240
Let's solve for the
optimal m and b.

00:00:14.240 --> 00:00:16.190
And just based on what we did
in the last videos, there's

00:00:16.190 --> 00:00:16.850
two ways to do that.

00:00:16.850 --> 00:00:19.180
We actually now know two points
that lie on that line.

00:00:19.180 --> 00:00:21.780
So we can literally find the
slope of that line and then

00:00:21.780 --> 00:00:23.700
the the y intercept,
the b there.

00:00:23.700 --> 00:00:25.230
Or, we could just say
it's the solution to

00:00:25.230 --> 00:00:26.200
this system of equations.

00:00:26.200 --> 00:00:28.140
And they're actually
mathematically equivalent.

00:00:28.140 --> 00:00:31.230
So let's solve for m first. And
if we want to solve for m,

00:00:31.230 --> 00:00:32.830
we want to cancel out the b's.

00:00:32.830 --> 00:00:34.810
So let me rewrite this top
equation just the way it's

00:00:34.810 --> 00:00:36.440
written over here.

00:00:36.440 --> 00:00:42.690
We have m times the mean of the
x squareds plus b times

00:00:42.690 --> 00:00:47.570
the mean of-- Actually,
we could even do

00:00:47.570 --> 00:00:49.160
it better than that.

00:00:49.160 --> 00:00:51.990
One step better than that is to,
based on the work we did

00:00:51.990 --> 00:00:55.860
in the last video, we can just
subtract this bottom equation

00:00:55.860 --> 00:00:57.530
from this top equation.

00:00:57.530 --> 00:00:58.570
So let me subtract it.

00:00:58.570 --> 00:00:59.590
Or let's add the negatives.

00:00:59.590 --> 00:01:02.030
So if I make this negative,
this is negative.

00:01:02.030 --> 00:01:03.060
This is negative.

00:01:03.060 --> 00:01:04.980
What do we get?

00:01:04.980 --> 00:01:12.630
We get m times the mean of the
x's minus the mean of the x

00:01:12.630 --> 00:01:15.730
squareds over the mean of x.

00:01:18.270 --> 00:01:20.760
The plus b and the negative
b cancel out.

00:01:20.760 --> 00:01:27.960
Is equal to the mean of the y's
minus the mean of the xy's

00:01:27.960 --> 00:01:30.780
over the mean of the x's.

00:01:30.780 --> 00:01:34.860
And then, we can divide both
sides of the equation by this.

00:01:34.860 --> 00:01:43.620
And so we get m is equal to the
mean of the y's minus the

00:01:43.620 --> 00:01:49.720
mean of the xy's over the mean
of the x's over this.

00:01:49.720 --> 00:01:53.600
The mean of the x's minus the
mean of the x squareds over

00:01:53.600 --> 00:01:55.150
the mean of the x's.

00:01:55.150 --> 00:01:58.220
Now notice, this is the exact
same thing that you would get

00:01:58.220 --> 00:02:01.760
if you found the slope between
these two points over here.

00:02:01.760 --> 00:02:05.380
Change in y, so the difference
between that y and that y, is

00:02:05.380 --> 00:02:06.640
that right over there.

00:02:06.640 --> 00:02:08.030
Over the change in x's.

00:02:08.030 --> 00:02:10.660
The change in that
x minus that x is

00:02:10.660 --> 00:02:12.160
exactly this over here.

00:02:12.160 --> 00:02:14.550
Now, to simplify it, we can
multiply both the numerator

00:02:14.550 --> 00:02:18.620
and the denominator by
the mean of the x's.

00:02:18.620 --> 00:02:20.370
And I do that just so we don't
have this in the denominator

00:02:20.370 --> 00:02:21.580
both places.

00:02:21.580 --> 00:02:23.840
So if we multiply the numerator
by the mean of the

00:02:23.840 --> 00:02:31.030
x's, we get the mean of the x's
times the mean of the y's

00:02:31.030 --> 00:02:34.280
minus, this and this will
cancel out, minus

00:02:34.280 --> 00:02:36.040
the mean of the xy's.

00:02:39.410 --> 00:02:42.400
All of that over, mean of the
x's times the mean of the x's

00:02:42.400 --> 00:02:52.170
is just going to be the mean of
the x's squared, minus over

00:02:52.170 --> 00:02:57.290
here you have the mean
of the x squared.

00:02:57.290 --> 00:03:01.010
And that's what we get for m.

00:03:01.010 --> 00:03:05.890
And if we want to solve for
b, we literally can just

00:03:05.890 --> 00:03:10.990
substitute back into either
equation, but this equation

00:03:10.990 --> 00:03:12.790
right here is simpler.

00:03:12.790 --> 00:03:16.115
And so if we wanted to solve for
b there, we can solve for

00:03:16.115 --> 00:03:17.510
b in terms of m.

00:03:17.510 --> 00:03:20.910
We just subtract m times the
mean of x's from both sides.

00:03:20.910 --> 00:03:30.320
We get b is equal to the mean
of the y's minus m times the

00:03:30.320 --> 00:03:31.730
mean of the x's.

00:03:31.730 --> 00:03:33.860
So what you do is you take
your data point.

00:03:33.860 --> 00:03:36.440
You find the mean of the x's,
the mean of the y's , the mean

00:03:36.440 --> 00:03:40.370
of the xy's, the mean
of the x's squared.

00:03:40.370 --> 00:03:41.730
You find your m.

00:03:41.730 --> 00:03:44.520
Once you find your m, then you
can substitute back in here

00:03:44.520 --> 00:03:45.790
and you find your b.

00:03:45.790 --> 00:03:49.130
And then you have your
actual optimal line.

00:03:49.130 --> 00:03:49.800
And we're done.

00:03:49.800 --> 00:03:53.100
So these are the two big formula
take aways for our

00:03:53.100 --> 00:03:54.690
optimal line.

00:03:54.690 --> 00:03:58.130
What I'm going to do in the next
video, and this is where

00:03:58.130 --> 00:04:00.730
if anyone wasn't skipping up to
this point, the next video

00:04:00.730 --> 00:04:02.360
is where they should re-engage,
because we're

00:04:02.360 --> 00:04:06.420
actually going to use these
formulas for the

00:04:06.420 --> 00:04:07.440
best fitting line.

00:04:07.440 --> 00:04:10.440
At least, when you measure the
error by the squared distances

00:04:10.440 --> 00:04:11.620
from the points.

00:04:11.620 --> 00:04:13.930
We're going to use these
formulas to actually find the

00:04:13.930 --> 00:04:16.780
best line for some data.