Sal Khan Interview with IT Conversations - January 31,2010

WEBVTT
Kind: captions
Language: en

00:00:00.656 --> 00:00:03.569
Sal Kahn's response to
the crisis in education

00:00:03.569 --> 00:00:06.143
is Kahnacademy.org, a site that lists

00:00:06.143 --> 00:00:07.991
a vast and growing collection of his

00:00:07.991 --> 00:00:10.486
YouTube video lessons in Math, Physics,

00:00:10.486 --> 00:00:12.983
Chemistry, Biology and Economics.

00:00:12.983 --> 00:00:15.349
In this conversation with
"Interviews With Innovators"

00:00:15.349 --> 00:00:17.548
host, Jon Udell, he discusses his

00:00:17.548 --> 00:00:19.767
teaching philosophies and methods and

00:00:19.767 --> 00:00:21.764
explains why he abandoned a career in

00:00:21.764 --> 00:00:24.748
financial services to become
a new kind of teacher ...

00:00:24.748 --> 00:00:26.849
From IT Conversations.

00:00:26.849 --> 00:00:38.148
(jazz music)

00:00:38.148 --> 00:00:40.355
Phil: Hi, and welcome to IT Conversations.

00:00:40.355 --> 00:00:42.355
I'm Phil Windley the Executive Producer.

00:00:42.355 --> 00:00:44.520
Today I'm happy to bring
you another program from

00:00:44.520 --> 00:00:46.849
Jon Udell's "Interviews With Innovators".

00:00:46.849 --> 00:00:49.271
This program is made
possible by Microsoft's

00:00:49.271 --> 00:00:51.232
Channel 9 and Channel 10.

00:00:51.232 --> 00:00:52.859
Voiceover: Our audio
files are delivered by

00:00:52.859 --> 00:00:55.620
Limelight Networks, the
high-performance content delivery

00:00:55.620 --> 00:00:57.271
network for digital media.

00:00:57.271 --> 00:01:00.850
The Conversations Network
is a 501(c)(3) non-profit

00:01:00.850 --> 00:01:02.133
and we need your help.

00:01:02.133 --> 00:01:04.525
For a tax-deductible
donation for as little as

00:01:04.525 --> 00:01:07.432
$5 per month, you can support this channel

00:01:07.432 --> 00:01:09.580
and the rest of the Conversations Network,

00:01:09.580 --> 00:01:12.078
so please visit: Conversationsnetwork.org

00:01:12.078 --> 00:01:14.161
to become a member and help us continue to

00:01:14.161 --> 00:01:16.944
bring our programs to the world for free.

00:01:16.944 --> 00:01:18.750
And now, here's John Udell.

00:01:18.750 --> 00:01:21.748
Jon: I saw something
online the other evening

00:01:21.748 --> 00:01:23.244
that reminded me of what you're doing

00:01:23.244 --> 00:01:25.750
in an interesting way, so it might be

00:01:25.750 --> 00:01:26.742
a fun place to start.

00:01:26.742 --> 00:01:30.420
There's an artist in Los Angeles named

00:01:30.420 --> 00:01:33.300
Richard Ankrom and somewhere around

00:01:33.300 --> 00:01:36.661
2000 or 2001, he got pissed
off about the fact that

00:01:36.661 --> 00:01:39.842
there's this freeway
intersection in Los Angeles

00:01:39.842 --> 00:01:42.163
that's very poorly labeled.

00:01:42.163 --> 00:01:44.343
In particular, there
was a key missing piece

00:01:44.343 --> 00:01:46.343
of information which was a place where

00:01:46.343 --> 00:01:49.164
there should have been a
sign that said "North 5".

00:01:49.164 --> 00:01:51.333
It just wasn't there.

00:01:51.333 --> 00:01:57.581
(laughs) So he went and
made a "North 5" sign

00:01:57.581 --> 00:02:00.844
to the Highway Department's
specifications,

00:02:00.844 --> 00:02:03.443
then he made himself a uniform,

00:02:03.443 --> 00:02:04.938
then he got up on a ladder and

00:02:04.938 --> 00:02:06.748
he went out and he installed the thing.

00:02:06.748 --> 00:02:10.820
(laughter)

00:02:10.820 --> 00:02:13.044
He calls this "Gorilla
Public Service" as in

00:02:13.044 --> 00:02:14.577
"Gorilla the Freedom Fighter", not

00:02:14.577 --> 00:02:17.243
"Gorilla the Great Ape",

00:02:17.243 --> 00:02:20.332
and since then, as far
as I know, it's still up

00:02:20.332 --> 00:02:23.500
and probably millions of
people have benefited from that

00:02:23.500 --> 00:02:26.495
and it just reminded
me of what you're doing

00:02:26.495 --> 00:02:29.544
in a way, in the realm
of education which is

00:02:29.544 --> 00:02:31.832
sort of looking at a
situation and thinking,

00:02:31.832 --> 00:02:32.998
"You know what? I'm just going to go do

00:02:32.998 --> 00:02:34.943
something about that."

00:02:34.943 --> 00:02:38.344
Thanks to the technology of the internet,

00:02:38.344 --> 00:02:40.282
I can be a super empowered individual,

00:02:40.282 --> 00:02:42.707
and there really is essentially no limit

00:02:42.707 --> 00:02:44.433
to how far that can scale out and

00:02:44.433 --> 00:02:46.784
how many people I can help.

00:02:46.784 --> 00:02:48.737
It's really a wonderful thing.

00:02:48.737 --> 00:02:52.335
So tell us about the Kahn Academy.

00:02:52.335 --> 00:02:57.541
Sal: It's most known for the
roughly 1,200 videos on YouTube.

00:02:57.541 --> 00:02:59.364
It's being aired other places and

00:02:59.364 --> 00:03:01.522
other people are distributing it offline,

00:03:01.522 --> 00:03:03.950
but YouTube's really the main way that

00:03:03.950 --> 00:03:05.865
people access the videos.

00:03:05.865 --> 00:03:09.952
It really started off very organically and

00:03:09.952 --> 00:03:12.537
I was helping cousins
and family members and

00:03:12.537 --> 00:03:14.861
it seemed to be working well.

00:03:14.861 --> 00:03:16.135
I was tutoring them remotely.

00:03:16.135 --> 00:03:18.837
I was in Boston, I'm in
Northern California now,

00:03:18.837 --> 00:03:21.828
but they were in New Orleans.

00:03:21.828 --> 00:03:23.537
When I was doing this
one-on-one tutoring with them,

00:03:23.537 --> 00:03:24.737
it seemed to be very effective.

00:03:24.737 --> 00:03:26.638
I just kept thinking of
how could I make it scale.

00:03:26.638 --> 00:03:29.037
There's actually a software piece that

00:03:29.037 --> 00:03:30.786
I started working on at first and

00:03:30.786 --> 00:03:31.371
a buddy of mine said,

00:03:31.371 --> 00:03:34.337
"Why don't you put some
videos in the software?"

00:03:34.337 --> 00:03:35.538
By the time I got over the idea that

00:03:35.538 --> 00:03:38.836
I didn't think of it
(laughs), I started recording

00:03:38.836 --> 00:03:42.536
a couple of videos and the
videos are what really took off.

00:03:42.536 --> 00:03:44.340
I think it's just the
low barrier for people

00:03:44.340 --> 00:03:46.128
to use it, even my own cousins say that

00:03:46.128 --> 00:03:47.836
I'm better on YouTube than I am live.

00:03:47.836 --> 00:03:49.197
Jon: (laughs)

00:03:49.197 --> 00:03:51.540
Sal: Pause and repeat
me and they don't like

00:03:51.540 --> 00:03:55.035
the video's judging them
the way I (laughs) ...

00:03:55.035 --> 00:03:59.927
I put it out there and I think as a fellow

00:03:59.927 --> 00:04:02.537
software engineer, you can
appreciate how fun it is

00:04:02.537 --> 00:04:04.418
when people actually
use something you built.

00:04:04.418 --> 00:04:06.451
When I just put it out there,

00:04:06.451 --> 00:04:08.521
I've written hundreds of
pieces of software that

00:04:08.521 --> 00:04:11.330
absolutely no one uses
and just put a couple

00:04:11.330 --> 00:04:12.865
YouTube videos out there and within a day,

00:04:12.865 --> 00:04:14.785
at least when I started,
20 or 30 people were

00:04:14.785 --> 00:04:18.036
using them and some of them
would write notes saying,

00:04:18.036 --> 00:04:20.277
"I got an A on my Algebra
exam because of this," or

00:04:20.277 --> 00:04:24.370
"I feel prepared to go
back to college next year."

00:04:24.370 --> 00:04:25.781
Some of them were, "I
was going to drop out

00:04:25.781 --> 00:04:27.835
until I saw something like this," so

00:04:27.835 --> 00:04:29.136
you just get one note
like that, you're like,

00:04:29.136 --> 00:04:30.927
"This has more impact than I've ever had

00:04:30.927 --> 00:04:32.698
doing anything else!" so I just kept

00:04:32.698 --> 00:04:35.635
making the videos and
the traffic just kept

00:04:35.635 --> 00:04:38.736
picking up organically
and it started to become

00:04:38.736 --> 00:04:41.635
pretty clear that there
was a need being addressed.

00:04:41.635 --> 00:04:45.138
Jon: So you've actually decided
to try to reconfigure your life,

00:04:45.138 --> 00:04:48.350
or you have reconfigured
your life around this,

00:04:48.350 --> 00:04:51.205
away from most recently,
financial services?

00:04:51.205 --> 00:04:54.138
Sal: Yeah. When I first
started doing this,

00:04:54.138 --> 00:04:56.335
I was originally tutoring my cousin Nadia

00:04:56.335 --> 00:04:58.279
in New Orleans, they had just visited me

00:04:58.279 --> 00:04:59.340
in Boston after my wedding.

00:04:59.340 --> 00:05:02.036
I was a hedge fund analyst in Boston and

00:05:02.036 --> 00:05:04.696
I was a hedge fund analyst until this past

00:05:04.696 --> 00:05:10.325
September and while my
day job was in finance,

00:05:10.325 --> 00:05:12.952
I had already reconfigured
my life around this.

00:05:12.952 --> 00:05:15.517
Pretty much every waking
moment where I felt that

00:05:15.517 --> 00:05:17.225
my wife wouldn't feel that I was being

00:05:17.225 --> 00:05:19.367
completely delinquent of my family duties,

00:05:19.367 --> 00:05:22.335
I would sneak away and
start doing YouTube videos.

00:05:22.335 --> 00:05:23.637
I'd also work on the software that would

00:05:23.637 --> 00:05:26.370
give people exercises
to do so I could track

00:05:26.370 --> 00:05:29.400
what they were doing.

00:05:29.400 --> 00:05:32.322
I've kind of just come to
terms with the fact that

00:05:32.322 --> 00:05:34.537
this is what I think I
should be doing all the time

00:05:34.537 --> 00:05:36.436
so in September I quit
my job and I've been

00:05:36.436 --> 00:05:39.131
doing it full time, and I
haven't looked back since,

00:05:39.131 --> 00:05:43.350
although my bank account
is slowly depleting,

00:05:43.350 --> 00:05:44.434
so hopefully I'll figure out a way to

00:05:44.434 --> 00:05:47.371
make that at least a "not deplete".

00:05:47.371 --> 00:05:50.367
Jon: I am certain that you will and

00:05:50.367 --> 00:05:53.287
I hope that people who
will be listening to this

00:05:53.287 --> 00:05:55.945
can maybe help with that because this is

00:05:55.945 --> 00:05:58.227
clearly what you should be spending your

00:05:58.227 --> 00:06:00.195
time doing and I think
you've written somewhere

00:06:00.195 --> 00:06:03.106
that you realized this
was the way you could

00:06:03.106 --> 00:06:06.036
have the most leverage
and do the most good

00:06:06.036 --> 00:06:07.356
and have the most fun.

00:06:07.356 --> 00:06:09.281
Sal: Oh yeah! Just think about it ...

00:06:09.281 --> 00:06:13.931
It's such a simple thing,
I could do a video on

00:06:13.931 --> 00:06:15.828
the Fundamental Theorem of Calculus and

00:06:15.828 --> 00:06:19.036
it might take me maybe if it's something

00:06:19.036 --> 00:06:20.943
I knew before I even started doing this,

00:06:20.943 --> 00:06:22.529
I don't have to prepare much and

00:06:22.529 --> 00:06:24.221
it takes me 10 or 20
minutes to do the video

00:06:24.221 --> 00:06:25.837
and I put it up there.

00:06:25.837 --> 00:06:28.436
Literally millions of people,

00:06:28.436 --> 00:06:29.734
for as long as the internet exists,

00:06:29.734 --> 00:06:32.137
can use that video.

00:06:32.137 --> 00:06:34.734
My or your great grandchildren
could use that video

00:06:34.734 --> 00:06:38.109
so just in terms of the
return on time invested

00:06:38.109 --> 00:06:40.946
or the return on mental capital invested,

00:06:40.946 --> 00:06:43.135
however you want to view
it, is mind-boggling.

00:06:43.135 --> 00:06:46.237
Jon: It's an astonishing
amount of leverage,

00:06:46.237 --> 00:06:48.737
but what this also points to is that,

00:06:48.737 --> 00:06:51.857
I was watching a ted talk that Bill Gates

00:06:51.857 --> 00:06:54.448
gave recently and part of the video was

00:06:54.448 --> 00:06:56.536
discussing education ... And you know from

00:06:56.536 --> 00:06:57.947
the software business, that there's this

00:06:57.947 --> 00:07:01.236
mime about there being
a really big difference

00:07:01.236 --> 00:07:04.277
between an average software engineer and

00:07:04.277 --> 00:07:06.857
a great software engineer; it's not just

00:07:06.857 --> 00:07:12.460
twice as good, it's [overdirect] magnitude

00:07:12.460 --> 00:07:14.136
difference, or at least this is the claim.

00:07:14.136 --> 00:07:16.224
He was making that same
argument in the realm

00:07:16.224 --> 00:07:18.691
of education and that the Gates Foundation

00:07:18.691 --> 00:07:21.274
had been looking into
this and had concluded

00:07:21.274 --> 00:07:22.186
that it's the same, right?

00:07:22.186 --> 00:07:25.636
That there are some teachers who are just

00:07:25.636 --> 00:07:28.635
way, way better than average and how

00:07:28.635 --> 00:07:31.135
can we leverage that?

00:07:31.135 --> 00:07:33.036
What you're doing is
a perfect illustration

00:07:33.036 --> 00:07:34.427
of how we could leverage that.

00:07:34.427 --> 00:07:36.536
I want to ask you about ...

00:07:36.536 --> 00:07:41.234
Clearly you're a great
teacher and you love to do it,

00:07:41.234 --> 00:07:44.444
but when did you realize that?

00:07:44.444 --> 00:07:47.932
What is it about you and how you think and

00:07:47.932 --> 00:07:52.537
how you present things
and how you communicate?

00:07:52.537 --> 00:07:54.838
Have you analyzed what it is that

00:07:54.838 --> 00:07:56.191
you think is effective?

00:07:56.191 --> 00:08:02.196
Sal: I've tried to. I
think about that myself.

00:08:02.196 --> 00:08:05.370
The idea of putting instruction on the web

00:08:05.370 --> 00:08:06.837
and letting people use it isn't new,

00:08:06.837 --> 00:08:08.280
I wasn't the first person to think of it,

00:08:08.280 --> 00:08:11.607
so I do introspect a lot as to why these

00:08:11.607 --> 00:08:13.945
videos have seemed to have struck a chord

00:08:13.945 --> 00:08:16.137
with people and gotten a lot of traction.

00:08:16.137 --> 00:08:17.734
I think it's a couple of things...

00:08:17.734 --> 00:08:20.415
You ask when did I realize it?

00:08:20.415 --> 00:08:24.316
I think this is true of a lot of people,

00:08:24.316 --> 00:08:25.230
it's probably true of yourself,

00:08:25.230 --> 00:08:26.143
or a lot of people who do well in the

00:08:26.143 --> 00:08:29.237
Math and Sciences, is
they kind of go beyond

00:08:29.237 --> 00:08:30.645
what's going on in the classroom and

00:08:30.645 --> 00:08:31.940
make connections on their own.

00:08:31.940 --> 00:08:33.338
If anything, they get frustrated by

00:08:33.338 --> 00:08:34.566
what goes on in the classroom.

00:08:34.566 --> 00:08:38.739
They see otherwise intelligent
peers memorizing facts

00:08:38.739 --> 00:08:41.830
and not really caring
about the actual intuition,

00:08:41.830 --> 00:08:43.238
and because they didn't
care about the intuition

00:08:43.238 --> 00:08:46.238
in their junior year, when
that same idea pops up

00:08:46.238 --> 00:08:49.738
in the senior year, it's like
they've never seen it before.

00:08:49.738 --> 00:08:53.071
It would boggle my mind,
you're relabeling the same

00:08:53.071 --> 00:08:54.641
concept over and over.

00:08:54.641 --> 00:08:55.835
I've always had the frustration.

00:08:55.835 --> 00:08:59.580
I've always been someone that always liked

00:08:59.580 --> 00:09:01.736
to ponder what I was
learning as opposed to,

00:09:01.736 --> 00:09:04.241
and sometimes it was
at my detriment because

00:09:04.241 --> 00:09:06.052
you only have so much time in your day

00:09:06.052 --> 00:09:07.636
and you can either do what the teacher

00:09:07.636 --> 00:09:09.236
tells you to do or you can ponder.

00:09:09.236 --> 00:09:10.835
The pondering might pay
off in the long term,

00:09:10.835 --> 00:09:13.138
but not doing what the
teacher tells you to do

00:09:13.138 --> 00:09:15.935
is what will hurt you in the short term.

00:09:15.935 --> 00:09:17.811
That was kind of my
frustration with my education

00:09:17.811 --> 00:09:20.033
throughout high school and college.

00:09:20.033 --> 00:09:21.836
I was kind of the guy
who'd skip class a lot

00:09:21.836 --> 00:09:24.303
and just try to read a
book, then read other books

00:09:24.303 --> 00:09:26.054
and talk to my friends about the concept

00:09:26.054 --> 00:09:28.138
as opposed to necessarily going to classes

00:09:28.138 --> 00:09:31.134
because I felt I was either lost in class

00:09:31.134 --> 00:09:33.641
or I was bored in class.

00:09:33.641 --> 00:09:35.561
On a good class, maybe
there'd be 10 minutes

00:09:35.561 --> 00:09:39.144
where it was catering
to me and so that was

00:09:39.144 --> 00:09:41.894
also a very frustrating experience.

00:09:41.894 --> 00:09:46.471
It really felt like there
must be a better way.

00:09:46.471 --> 00:09:49.835
When I started tutoring my cousin Nadia,

00:09:49.835 --> 00:09:53.473
I started tutoring her
as a friend, or a cousin

00:09:53.473 --> 00:09:55.557
as opposed to a teacher.

00:09:55.557 --> 00:09:58.731
I think that tone, if
I had to pick one thing

00:09:58.731 --> 00:10:01.735
that has kind of resonated with people,

00:10:01.735 --> 00:10:03.057
it's this tone that ...

00:10:03.057 --> 00:10:05.139
I got a note the other
day on the YouTube Channel

00:10:05.139 --> 00:10:07.476
where, "You teach more like
a friend than a teacher."

00:10:07.476 --> 00:10:09.224
Jon: Yeah, conversational tone.

00:10:09.224 --> 00:10:10.557
Sal: You interpret that in multiple ways,

00:10:10.557 --> 00:10:13.552
but I think that's the core issue ...

00:10:13.552 --> 00:10:15.837
That the tone doesn't turn people off.

00:10:15.837 --> 00:10:19.145
The next thing what I touched
on in my own experience

00:10:19.145 --> 00:10:21.356
is that I'm really big
on making connections.

00:10:21.356 --> 00:10:23.447
I get a lot of letters
and you can almost sense

00:10:23.447 --> 00:10:26.047
the anger from the students where they

00:10:26.047 --> 00:10:28.355
communicate that their teachers think

00:10:28.355 --> 00:10:30.274
they're making it easier
for the students by

00:10:30.274 --> 00:10:33.188
not giving the intuition,
by not giving the depth

00:10:33.188 --> 00:10:34.857
or making the connections because they

00:10:34.857 --> 00:10:36.345
kind of think that the
students just need to

00:10:36.345 --> 00:10:37.851
learn the mechanics.

00:10:37.851 --> 00:10:39.648
I get letter after letter saying because

00:10:39.648 --> 00:10:40.446
you get the intuition,

00:10:40.446 --> 00:10:42.855
because you give the common sense

00:10:42.855 --> 00:10:46.019
why this is just a mathematical way of

00:10:46.019 --> 00:10:47.741
expressing something that
makes a lot of sense,

00:10:47.741 --> 00:10:49.648
it actually becomes a lot more easy and

00:10:49.648 --> 00:10:51.609
a lot more intuitive.

00:10:51.609 --> 00:10:53.946
I think if you combine that with ...

00:10:53.946 --> 00:10:56.769
YouTube used to limit me to 10 minutes and

00:10:56.769 --> 00:10:58.778
I've gotten letters from
researchers telling me

00:10:58.778 --> 00:11:00.547
that it's been established that 9 minutes

00:11:00.547 --> 00:11:02.447
is about as long as
someone can really focus

00:11:02.447 --> 00:11:05.746
and pay attention without
taking a mental break.

00:11:05.746 --> 00:11:07.046
I give credit to YouTube for that.

00:11:07.046 --> 00:11:10.047
I now can record a little
bit longer than 10 minutes,

00:11:10.047 --> 00:11:11.451
but I think that's a big thing.

00:11:11.451 --> 00:11:12.946
The other thing, and
this was just a complete

00:11:12.946 --> 00:11:15.490
side effect of me trying to do it as

00:11:15.490 --> 00:11:17.104
cheaply as possible without fancy video

00:11:17.104 --> 00:11:19.446
equipment is, you don't see my face and

00:11:19.446 --> 00:11:21.194
I've been told by a lot of people that

00:11:21.194 --> 00:11:23.361
that keeps it from being a distraction.

00:11:23.361 --> 00:11:24.737
It actually makes it very easy for me to

00:11:24.737 --> 00:11:25.947
make the videos, I don't have to shave or

00:11:25.947 --> 00:11:29.447
wash my face (laughter)
before I make every video.

00:11:29.447 --> 00:11:32.688
I just have to make sure ...

00:11:32.688 --> 00:11:37.646
Jon: The process that you use is
apparently delightfully low-tech.

00:11:37.646 --> 00:11:41.359
It looks to me like you're using tools

00:11:41.359 --> 00:11:45.194
as simple as Windows Paint
and I don't know what,

00:11:45.194 --> 00:11:46.328
but it's nothing fancy, right?

00:11:46.328 --> 00:11:49.611
Sal: No, the first time where I decided to

00:11:49.611 --> 00:11:51.250
make the video, I said, "How can I make a

00:11:51.250 --> 00:11:53.109
decent video for my
cousins?" and I literally

00:11:53.109 --> 00:11:57.341
took Microsoft Paint,
made the background black,

00:11:57.341 --> 00:11:58.248
just because I thought it would look nice,

00:11:58.248 --> 00:12:01.547
like a chalkboard, and
I used some shareware

00:12:01.547 --> 00:12:03.773
screen capture software, it was like a

00:12:03.773 --> 00:12:05.146
"free for 20 days" and then you gotta pay

00:12:05.146 --> 00:12:09.548
$20 after that, I got myself a $20 headset

00:12:09.548 --> 00:12:14.153
and $60 Wacom Pen Tablet
and I started doing it.

00:12:14.153 --> 00:12:16.262
Actually, the production
quality was pretty good.

00:12:16.262 --> 00:12:19.430
The first, I would say,
4 or 500 videos were done

00:12:19.430 --> 00:12:20.767
exactly like that and you can see,

00:12:20.767 --> 00:12:22.144
if you see some of the earlier videos,

00:12:22.144 --> 00:12:25.144
they're now upgraded slightly,
but not really that much.

00:12:25.144 --> 00:12:28.350
One of the users just sent me a license

00:12:28.350 --> 00:12:29.849
to Camptasia; I think there's
other things I could use

00:12:29.849 --> 00:12:33.844
as well, that would do
screen capture fairly well.

00:12:33.844 --> 00:12:37.263
I now use a slightly better
art program that at least

00:12:37.263 --> 00:12:39.496
it's a little bit sensitive
to pressure on the

00:12:39.496 --> 00:12:41.883
pen tablet, but it's
still pretty low-tech.

00:12:41.883 --> 00:12:44.633
Jon: Yeah, exactly.

00:12:44.633 --> 00:12:48.133
It's interesting because
when I got started,

00:12:48.133 --> 00:12:51.710
I was actually, if you look on Wikipedia,

00:12:51.710 --> 00:12:53.967
I'm the person whose
name is associated with

00:12:53.967 --> 00:12:56.032
the word "Screencast"
because I got interested

00:12:56.032 --> 00:12:59.932
in that stuff some years ago and actually

00:12:59.932 --> 00:13:02.318
ran a little ... Because
I was making these

00:13:02.318 --> 00:13:06.132
software demos using
Camptasia and its predecessors

00:13:06.132 --> 00:13:10.144
and got real interested in that process.

00:13:10.144 --> 00:13:12.551
At a certain point, I got to where

00:13:12.551 --> 00:13:16.249
I was making pretty
highly-produced kinds of things

00:13:16.249 --> 00:13:18.661
that were tightly edited
and that was interesting

00:13:18.661 --> 00:13:22.328
to do that process, but at a later point,

00:13:22.328 --> 00:13:26.250
I started to rethink
that and at one moment

00:13:26.250 --> 00:13:28.447
I remember a friend of mine, his name is

00:13:28.447 --> 00:13:33.248
Chris [Gemignani] and
he's an Analytics and

00:13:33.248 --> 00:13:36.254
Visualization person,
and every once in a while

00:13:36.254 --> 00:13:39.548
he makes these screencasts to show people

00:13:39.548 --> 00:13:44.498
things that he can teach
them in say, Excel.

00:13:44.498 --> 00:13:47.663
He made one of these and it was full of

00:13:47.663 --> 00:13:50.658
stops and starts and you know,

00:13:50.658 --> 00:13:52.750
"Oh, I didn't mean to do it that way," and

00:13:52.750 --> 00:13:55.351
"Let's back up and try another path," and

00:13:55.351 --> 00:13:58.440
we talked about it afterwards and he said,

00:13:58.440 --> 00:14:00.549
"You know, I felt like I
should edit that stuff,

00:14:00.549 --> 00:14:03.448
but then I realized in a way, if I did,

00:14:03.448 --> 00:14:05.658
I wouldn't be giving
people their true insight

00:14:05.658 --> 00:14:07.911
into the process of how I think about

00:14:07.911 --> 00:14:11.490
these things and how I do these things."

00:14:11.490 --> 00:14:14.546
In fact, the unedited style wound up being

00:14:14.546 --> 00:14:17.380
better and I think maybe
that's true for you, too.

00:14:17.380 --> 00:14:21.046
Sal: Oh absolutely. That's probably one of

00:14:21.046 --> 00:14:22.883
the top three in terms of why ...

00:14:22.883 --> 00:14:24.880
It was really out of laziness initially.

00:14:24.880 --> 00:14:25.879
Jon: (laughs)

00:14:25.879 --> 00:14:27.148
Sal: It was for my cousins, I was like,

00:14:27.148 --> 00:14:29.338
"I don't care what they
think. I'm doing this

00:14:29.338 --> 00:14:32.146
out of my own ... ," so
I would put it out there.

00:14:32.146 --> 00:14:34.835
I wouldn't do the problem
before I made the video,

00:14:34.835 --> 00:14:38.648
so they saw me doing the
problems the first time.

00:14:38.648 --> 00:14:40.648
Every now and then, I
would go down one route

00:14:40.648 --> 00:14:42.848
and say, "Wait, that's
not right," and I would

00:14:42.848 --> 00:14:44.146
back up a little bit.

00:14:44.146 --> 00:14:47.248
That's something that
frankly, no one ever sees.

00:14:47.248 --> 00:14:50.148
Your lecturer in your
high school Trigonometry

00:14:50.148 --> 00:14:52.555
class, they have done the
problem ahead of time,

00:14:52.555 --> 00:14:54.553
or they have the solution
in front of them.

00:14:54.553 --> 00:14:55.646
You just see the finished product,

00:14:55.646 --> 00:14:57.245
you don't actually see
the thought process.

00:14:57.245 --> 00:14:58.746
I think to actually see
someone stumble around

00:14:58.746 --> 00:15:01.468
a little bit, who you
respect, who you realize

00:15:01.468 --> 00:15:03.377
knows what they're doing,
but still they stumble

00:15:03.377 --> 00:15:04.831
around and see kind of the art of solving

00:15:04.831 --> 00:15:06.894
a problem I think is
something you'll never see

00:15:06.894 --> 00:15:09.054
in a traditional lecture or textbook.

00:15:09.054 --> 00:15:11.510
Jon: That might also imply
that you do relatively

00:15:11.510 --> 00:15:14.142
less prep than another
person might imagine

00:15:14.142 --> 00:15:17.755
would be necessary because
thinking on your feet

00:15:17.755 --> 00:15:19.254
is part of the process.

00:15:19.254 --> 00:15:21.644
Sal: Now I kind of
embrace that idea now that

00:15:21.644 --> 00:15:23.841
there's more viewership,
I feel like I should

00:15:23.841 --> 00:15:25.528
prepare more and make it more polished,

00:15:25.528 --> 00:15:29.856
but I kind of fight that
instinct and I still

00:15:29.856 --> 00:15:31.510
want to keep them relatively raw.

00:15:31.510 --> 00:15:33.178
I want to make them easy to watch,

00:15:33.178 --> 00:15:38.175
but I get a lot of letters,
people appreciate these ...

00:15:38.175 --> 00:15:40.241
There's the college board SAT book and

00:15:40.241 --> 00:15:41.937
I did all 400 Math
problems and I literally

00:15:41.937 --> 00:15:43.545
did them in real time so ...

00:15:43.545 --> 00:15:45.260
Literally like sitting
on my shoulder while

00:15:45.260 --> 00:15:47.346
I take an SAT exam which is a kind of,

00:15:47.346 --> 00:15:50.336
I don't think a lot of people will expose

00:15:50.336 --> 00:15:54.870
themselves in that way,
but I think it's an

00:15:54.870 --> 00:15:55.522
incredibly valuable experience because you

00:15:55.522 --> 00:15:58.545
don't actually get to
see that in real life.

00:15:58.545 --> 00:16:00.845
Jon: In so many ways, we
never get the opportunity

00:16:00.845 --> 00:16:04.841
to look over the shoulder
of a practitioner,

00:16:04.841 --> 00:16:06.231
of any discipline.

00:16:06.231 --> 00:16:09.512
This has been one of my biggest hopes for

00:16:09.512 --> 00:16:12.141
the medium of internet
video, as a matter of fact,

00:16:12.141 --> 00:16:15.674
is that exactly that becomes possible.

00:16:15.674 --> 00:16:22.142
I've thought a lot about how in earlier

00:16:22.142 --> 00:16:25.510
incarnations of human society,
that was really the norm.

00:16:25.510 --> 00:16:30.677
When you were living
in tribal or a village

00:16:30.677 --> 00:16:34.542
kind of situation, the
work that people did

00:16:34.542 --> 00:16:36.443
was visible to you, right?

00:16:36.443 --> 00:16:38.534
You just had to look and
see what people were doing.

00:16:38.534 --> 00:16:40.543
It was just out there in the open and so

00:16:40.543 --> 00:16:45.340
part of education was just
seeing what people did.

00:16:45.340 --> 00:16:47.755
In a funny kind of way,
I realized that when

00:16:47.755 --> 00:16:49.443
got to this mode where
we were all in cubicles

00:16:49.443 --> 00:16:54.343
with keyboards and screens,
but were communicating

00:16:54.343 --> 00:16:55.943
through email and communicating through

00:16:55.943 --> 00:16:59.430
these other sort of mediated channels,

00:16:59.430 --> 00:17:01.988
in a way, we actually lose the ability to

00:17:01.988 --> 00:17:03.835
watch other people, to
see how they work with

00:17:03.835 --> 00:17:05.942
their tools to see how
their minds work, right?

00:17:05.942 --> 00:17:10.404
So like with capturing that
is a really interesting

00:17:10.404 --> 00:17:13.154
possibility for the
medium of internet video,

00:17:13.154 --> 00:17:16.430
and you're a great illustration of that.

00:17:16.430 --> 00:17:18.743
Sal: I 100% agree and I think that extends

00:17:18.743 --> 00:17:20.432
even beyond what I'm doing.

00:17:20.432 --> 00:17:25.459
I won't name him because the story isn't

00:17:25.459 --> 00:17:27.544
flattering to his
company, but he's kind of

00:17:27.544 --> 00:17:29.454
used my example and he does analytics

00:17:29.454 --> 00:17:31.935
at a company and he was
running some reports.

00:17:31.935 --> 00:17:33.736
Instead of writing a white paper on what

00:17:33.736 --> 00:17:35.955
this report means or how do you run it,

00:17:35.955 --> 00:17:38.540
like you would traditionally
do in a corporation,

00:17:38.540 --> 00:17:39.835
he told the CEO of the company,

00:17:39.835 --> 00:17:41.704
"I have this buddy, he
does this thing on YouTube

00:17:41.704 --> 00:17:43.721
using Camptasia. How
about I do the same thing?

00:17:43.721 --> 00:17:45.380
Instead of running a white
paper on how to do it,

00:17:45.380 --> 00:17:47.235
how about I just capture
my screen while I do it

00:17:47.235 --> 00:17:48.433
and I'll talk out loud and ... ."

00:17:48.433 --> 00:17:50.636
I thought it was a brilliant idea.

00:17:50.636 --> 00:17:52.334
What better way than the guy who's kind of

00:17:52.334 --> 00:17:54.737
writing it to learn from
looking over his shoulder

00:17:54.737 --> 00:17:58.136
how he does it, but she
thought it was silly.

00:17:58.136 --> 00:18:00.540
Just because I think
it's not how the other

00:18:00.540 --> 00:18:02.463
corporations do it or it's not how the

00:18:02.463 --> 00:18:04.374
corporation she worked for did it.

00:18:04.374 --> 00:18:05.458
I think it's going to happen.

00:18:05.458 --> 00:18:08.335
I think a lot of these
short technical papers

00:18:08.335 --> 00:18:10.460
are going to become
obsolete because people

00:18:10.460 --> 00:18:11.873
can just look over someone's shoulder

00:18:11.873 --> 00:18:13.330
and see how they do it in real time.

00:18:13.330 --> 00:18:16.948
Jon: There's a video
that you may have seen by

00:18:16.948 --> 00:18:19.779
this guy, Hans Rosling.
He wrote the Gapminder

00:18:19.779 --> 00:18:23.192
software which is this
pretty incredible data

00:18:23.192 --> 00:18:24.864
visualization, he's given
a couple of Ted talks,

00:18:24.864 --> 00:18:29.236
and what you realize
when you see him giving

00:18:29.236 --> 00:18:33.236
one of these presentations,
is that it's only

00:18:33.236 --> 00:18:37.446
partly about the high quality
animated visualizations

00:18:37.446 --> 00:18:39.858
which the software can produce.

00:18:39.858 --> 00:18:43.727
It's also about how he performs

00:18:43.727 --> 00:18:48.356
the presentation around that, so it's the

00:18:48.356 --> 00:18:51.736
combination of him using the software

00:18:51.736 --> 00:18:53.734
along with his voice, along with his body

00:18:53.734 --> 00:18:56.126
in this case to illustrate.

00:18:56.126 --> 00:18:58.135
That is incredibly powerful.

00:18:58.550 --> 00:19:04.356
What I wanted to get to, though, was where

00:19:04.356 --> 00:19:08.533
are the limitations still
and how can we address

00:19:08.533 --> 00:19:10.934
them ... so one thing a person doesn't

00:19:10.934 --> 00:19:13.318
get to do is interact with you.

00:19:13.318 --> 00:19:17.443
It's one way in the
course of these videos,

00:19:17.443 --> 00:19:24.359
so it's not sort of a complete replacement

00:19:24.359 --> 00:19:27.633
for much or some of
what happens in school,

00:19:27.633 --> 00:19:30.632
it's however, a useful
compliment and I have

00:19:30.632 --> 00:19:35.939
read some indications
that people are getting to

00:19:35.939 --> 00:19:38.447
combine your stuff with classrooms in

00:19:38.447 --> 00:19:40.358
interactive settings.

00:19:40.358 --> 00:19:42.532
So what do you think about that?

00:19:42.532 --> 00:19:46.032
Sal: When I started this,
that was kind of my sense;

00:19:46.032 --> 00:19:47.944
it doesn't have the interactivity of

00:19:47.944 --> 00:19:53.228
a live one-on-one session,
but I've come to the

00:19:53.228 --> 00:19:55.443
conclusion, I don't
want to be too grandiose

00:19:55.443 --> 00:19:59.693
about its potential,
but I do think that it's

00:19:59.693 --> 00:20:03.034
hands-down better than
the professor sitting

00:20:03.034 --> 00:20:05.280
in front of a classroom
of 300 students and

00:20:05.280 --> 00:20:08.949
just lecturing a prepared
scripted lecture to them.

00:20:08.949 --> 00:20:12.634
One, that's kind of
artificial, it's a 90 minute

00:20:12.634 --> 00:20:15.233
long thing, it's probably
not catering to all

00:20:15.233 --> 00:20:16.777
of the students, some of
the students are lost,

00:20:16.777 --> 00:20:19.934
some are bored, they might be sleepy,

00:20:19.934 --> 00:20:20.939
it's just not the time ...

00:20:20.939 --> 00:20:22.864
Jon: There's no question about that.

00:20:22.864 --> 00:20:24.532
Sal: So here you're getting
it when you want it,

00:20:24.532 --> 00:20:26.784
you're getting it in granular chunks,

00:20:26.784 --> 00:20:30.134
and you can pause, repeat
if I say "orthogonal"

00:20:30.134 --> 00:20:31.641
and you don't know what that means,

00:20:31.641 --> 00:20:34.043
you can pause and look it up on Wikipedia,

00:20:34.043 --> 00:20:36.336
you can go look at the
other video that I've done

00:20:36.336 --> 00:20:39.045
on orthogonality and fill in
the blanks just to review.

00:20:39.045 --> 00:20:41.882
I think on a lot of levels,
like my cousin said,

00:20:41.882 --> 00:20:46.515
I'm better in a lot of
lectures ... Would be better on

00:20:46.515 --> 00:20:48.432
YouTube than live.

00:20:48.432 --> 00:20:51.631
Then there's also the
opportunity to really

00:20:51.631 --> 00:20:54.500
tweak it where YouTube
gives me data on where

00:20:54.500 --> 00:20:55.874
attention span drops off from my videos,

00:20:55.874 --> 00:20:57.886
so if something unusual
happens in one of my

00:20:57.886 --> 00:20:59.964
videos, I can go back to that point,

00:20:59.964 --> 00:21:02.135
see what happened, maybe
record another video

00:21:02.135 --> 00:21:04.480
if it's because I threw
out a weird concept

00:21:04.480 --> 00:21:05.735
or I skipped a step or whatever.

00:21:05.735 --> 00:21:06.542
Jon: Exactly.

00:21:06.542 --> 00:21:09.728
Sal: So there's also this
potential of really tweeking.

00:21:09.728 --> 00:21:11.827
That's why I want to do this software,

00:21:11.827 --> 00:21:13.310
and I've been working on the software,

00:21:13.310 --> 00:21:14.709
because I want to see
how people interact with

00:21:14.709 --> 00:21:17.334
exercises before and after
the video interaction.

00:21:17.334 --> 00:21:19.715
Attention span is one thing,
but I also want to see

00:21:19.715 --> 00:21:21.433
whether they learned.

00:21:21.433 --> 00:21:24.242
You can imagine a world
where you make a video

00:21:24.242 --> 00:21:25.738
or I make a video or we
make multiple versions

00:21:25.738 --> 00:21:27.734
of the same video and
we just see which ones

00:21:27.734 --> 00:21:29.160
work best for different people.

00:21:29.160 --> 00:21:30.816
Jon: Absolutely.

00:21:30.816 --> 00:21:32.132
Sal: There's things that have never been

00:21:32.132 --> 00:21:33.831
imagined with traditional.

00:21:33.831 --> 00:21:36.310
The limitation I think,
is just that notion that

00:21:36.310 --> 00:21:39.310
there's this individual who
cares directly about you.

00:21:39.310 --> 00:21:40.533
When I worked first
worked with my cousins,

00:21:40.533 --> 00:21:43.076
I think it's that idea
of my 12-year-old cousin

00:21:43.076 --> 00:21:47.239
feeling like her, at the
time I was 26-year-old,

00:21:47.239 --> 00:21:50.718
cousin who took time out
of his day to mentor her.

00:21:50.718 --> 00:21:52.658
I think that's incredibly,
incredibly valuable.

00:21:52.658 --> 00:21:55.654
Frankly, that's not going on in the school

00:21:55.654 --> 00:21:56.510
system right now, right?

00:21:56.510 --> 00:21:58.432
Right now you're lucky if
you have a conversation

00:21:58.432 --> 00:22:01.511
with your teacher in
a class of 30 and it's

00:22:01.511 --> 00:22:03.738
usually the problem kids who are having

00:22:03.738 --> 00:22:05.131
the most time with the
teacher and it's not

00:22:05.131 --> 00:22:06.744
a positive interaction.

00:22:06.744 --> 00:22:09.655
Jon: I'll tell you a
funny story about some

00:22:09.655 --> 00:22:13.840
other educational videos that I've seen.

00:22:13.840 --> 00:22:17.231
I catch a lot of this
stuff in podcast form

00:22:17.231 --> 00:22:21.432
and at one point I was
kind of auditing a course

00:22:21.432 --> 00:22:26.233
on Evolutionary Biology out
of, I think it was Berkley,

00:22:26.233 --> 00:22:29.431
so I remembered actually
something I had seen

00:22:29.431 --> 00:22:33.248
in that course a couple
of years before and

00:22:33.248 --> 00:22:35.670
I wanted to use that
insight in something I was

00:22:35.670 --> 00:22:37.434
writing on my blog.

00:22:37.434 --> 00:22:41.521
I went back to try to find it and

00:22:41.521 --> 00:22:42.673
when I went back to look at it,

00:22:42.673 --> 00:22:45.433
I wound up doing the video instead of

00:22:45.433 --> 00:22:48.231
the audio like I usually
do, and my daughter,

00:22:48.231 --> 00:22:49.934
who was just getting
ready to enter college,

00:22:49.934 --> 00:22:52.667
was looking over my shoulder while I was

00:22:52.667 --> 00:22:54.750
playing this video and she said,

00:22:54.750 --> 00:22:57.433
"So, do the students just sit there?"

00:22:57.433 --> 00:22:59.755
(laughter)

00:22:59.755 --> 00:23:02.751
Jon: I was like, "Yeah." In a
large lecture, they do, right?

00:23:02.751 --> 00:23:04.733
But then the other weird
thing about that was

00:23:04.733 --> 00:23:08.732
that when I finally found
that installment of the

00:23:08.732 --> 00:23:12.673
course, he wasn't as good
... What they had done

00:23:12.673 --> 00:23:15.734
is they had replaced
this semester's version

00:23:15.734 --> 00:23:20.233
with the version from two
years ago that I had seen.

00:23:20.233 --> 00:23:23.131
He'd been better two years ago.

00:23:23.131 --> 00:23:26.435
It was like he had the
more effective version

00:23:26.435 --> 00:23:29.132
so why not just can
that and reuse that and

00:23:29.132 --> 00:23:32.588
use the class time for supplementary

00:23:32.588 --> 00:23:35.032
kinds of things, or
interactive kinds of things.

00:23:35.032 --> 00:23:36.724
Sal: I've got several other teachers,

00:23:36.724 --> 00:23:39.840
they're starting to assign
the videos instead of

00:23:39.840 --> 00:23:41.218
reading and homework and then you do your

00:23:41.218 --> 00:23:42.340
homework in class.

00:23:42.340 --> 00:23:44.036
That's where you really benefit from;

00:23:44.036 --> 00:23:45.670
one, having the teacher there, even more,

00:23:45.670 --> 00:23:46.637
having your peers there.

00:23:46.637 --> 00:23:50.923
Jon: Exactly. I think that
will be really fruitful.

00:23:50.923 --> 00:23:54.332
Something else that I'm interested in is

00:23:54.332 --> 00:23:56.531
the notion that you
talked about caring about

00:23:56.531 --> 00:24:00.153
the intuition and
transmitting the intuition ...

00:24:00.153 --> 00:24:04.227
That happens in a pretty
personal way for you,

00:24:04.227 --> 00:24:08.387
so what we get is this
view through your eyes

00:24:08.387 --> 00:24:11.427
of how these intuitions made sense to you,

00:24:11.427 --> 00:24:14.934
which is great because a lot of times that

00:24:14.934 --> 00:24:17.055
doesn't come front and center.

00:24:17.055 --> 00:24:20.333
If we start to get a
few more people like you

00:24:20.333 --> 00:24:23.133
out there as well and we
can start to triangulate

00:24:23.133 --> 00:24:25.967
on these things, it'll
be even better because

00:24:25.967 --> 00:24:30.135
ultimately, there might
be some other person

00:24:30.135 --> 00:24:32.433
like you might have a
way of thinking about it

00:24:32.433 --> 00:24:34.138
that works better for me ...

00:24:34.138 --> 00:24:34.820
Sal: Absolutely.

00:24:34.820 --> 00:24:36.473
Jon: And if I could
access the context of a

00:24:36.473 --> 00:24:41.231
particular subject, if
I could triangulate and

00:24:41.231 --> 00:24:43.635
assemble, "Well, here's
Sal's way of thinking

00:24:43.635 --> 00:24:44.834
about it and here's somebody else's way

00:24:44.834 --> 00:24:47.231
of thinking about it,
if I could pull these

00:24:47.231 --> 00:24:50.531
things together ... " which you know,

00:24:50.531 --> 00:24:54.140
just through tagging
things on services like

00:24:54.140 --> 00:24:57.117
YouTube wouldn't be
that hard to let happen

00:24:57.117 --> 00:24:58.973
in a pretty loosely coupled way.

00:24:58.973 --> 00:25:02.432
Sal: Oh yeah, just from
people reviewing it or

00:25:02.432 --> 00:25:04.732
tagging it, you could
start to understand what

00:25:04.732 --> 00:25:06.731
will respond with
different people or maybe

00:25:06.731 --> 00:25:09.718
people could use multiple viewpoints,

00:25:09.718 --> 00:25:11.884
but then if you really do couple that with

00:25:11.884 --> 00:25:14.634
exercises where you're
able to track everything

00:25:14.634 --> 00:25:15.932
and do some analytics,
then you can do some

00:25:15.932 --> 00:25:19.431
pretty profound and really
scientifically determine

00:25:19.431 --> 00:25:21.332
what works for different people.

00:25:21.332 --> 00:25:24.332
Jon: Right. So say a little bit more about

00:25:24.332 --> 00:25:27.139
the software you're
working on, what it will do

00:25:27.139 --> 00:25:29.200
and where it's going.

00:25:29.200 --> 00:25:30.474
Sal: It already exists on
the side, I've actually

00:25:30.474 --> 00:25:34.624
just had to [port it and app end] because

00:25:34.624 --> 00:25:36.834
too many people were using it which is not

00:25:36.834 --> 00:25:39.386
a bad problem, but when
you're the support staff

00:25:39.386 --> 00:25:42.140
and engineering staff ...

00:25:42.140 --> 00:25:45.806
The idea's very simple;
it starts at 1 + 1 = 2,

00:25:45.806 --> 00:25:48.534
it just generates random problems and

00:25:48.534 --> 00:25:51.135
each module is like a concept,

00:25:51.135 --> 00:25:54.304
like basic addition; 1
+ 1 = 2, 2 + 3 = 5 ...

00:25:54.304 --> 00:25:55.932
You got to get 10 in a row.

00:25:55.932 --> 00:25:58.146
Once you have 10 in a row on that,

00:25:58.146 --> 00:26:00.437
and that's something that doesn't

00:26:00.437 --> 00:26:01.332
happen in the school system,

00:26:01.332 --> 00:26:02.641
no one expects you to get 10 in a row.

00:26:02.641 --> 00:26:05.055
They expect you to get
70% and then you move on.

00:26:05.055 --> 00:26:07.890
You get 10 in a row and then it's kind of,

00:26:07.890 --> 00:26:09.390
if you can imagine a no-directed graph,

00:26:09.390 --> 00:26:11.234
I have something called
a "Knowledge Map" and

00:26:11.234 --> 00:26:13.148
your basic addition,
once you get 10 in a row,

00:26:13.148 --> 00:26:14.970
you've then fulfilled
the prerequisites for

00:26:14.970 --> 00:26:17.723
Level 2 Addition and Basic Subtraction.

00:26:17.723 --> 00:26:19.932
Those fulfill the
prerequisites and you have this

00:26:19.932 --> 00:26:23.469
no-directed graph, right
now it goes up through

00:26:23.469 --> 00:26:26.558
pretty much all of Algebra
2 and Trigonometry.

00:26:26.558 --> 00:26:29.433
The videos on YouTube go well beyond that.

00:26:29.433 --> 00:26:31.967
They go into college-level
Math, Physics and

00:26:31.967 --> 00:26:35.232
Chemistry, but the software,
the idea is to have

00:26:35.232 --> 00:26:37.140
it at least cover everything
you would have to know

00:26:37.140 --> 00:26:41.233
for the SAT, the ACT or take the GED.

00:26:41.233 --> 00:26:45.332
The problems are randomly generated,

00:26:45.332 --> 00:26:48.833
so if it takes you 500
problems to understand

00:26:48.833 --> 00:26:50.473
negative exponents, you're going to get

00:26:50.473 --> 00:26:52.055
500 problems and in the process,

00:26:52.055 --> 00:26:53.832
it's tracking everything
you do and it knows when ...

00:26:53.832 --> 00:26:56.226
The videos are coupled into it so if you

00:26:56.226 --> 00:26:58.332
get a problem, if you know how to do it,

00:26:58.332 --> 00:26:59.637
you can type in the answer.

00:26:59.637 --> 00:27:00.468
If you don't know how to do it,

00:27:00.468 --> 00:27:01.532
you can click "Hint", it'll give you the

00:27:01.532 --> 00:27:03.632
steps for that exact problem.

00:27:03.632 --> 00:27:04.731
If you still don't know how to do it,

00:27:04.731 --> 00:27:06.334
you can click "Video" and the video from

00:27:06.334 --> 00:27:07.432
YouTube will pipe in.

00:27:07.432 --> 00:27:10.537
What I kind of envision
doing is closing the

00:27:10.537 --> 00:27:13.834
loop between instruction,
practice and assessment

00:27:13.834 --> 00:27:15.554
and making them all happen simultaneously

00:27:15.554 --> 00:27:17.435
as opposed to the traditional model where

00:27:17.435 --> 00:27:19.642
I lecture to you, I
give you homework which

00:27:19.642 --> 00:27:21.472
you do in a complete
vacuum without any feedback

00:27:21.472 --> 00:27:22.971
because the textbooks aren't giving you

00:27:22.971 --> 00:27:24.335
any feedback and you may or may not know

00:27:24.335 --> 00:27:26.976
how to do it, and then
we kind of keep going

00:27:26.976 --> 00:27:28.933
in that cycle for a week or two and then

00:27:28.933 --> 00:27:31.053
I give you an exam, you get a C,

00:27:31.053 --> 00:27:32.736
so you clearly have some deficiencies,

00:27:32.736 --> 00:27:34.932
but too bad, we have to
move onto the next concept.

00:27:34.932 --> 00:27:39.137
The model here is just close the loop and

00:27:39.137 --> 00:27:41.558
don't let anyone have any deficiencies

00:27:41.558 --> 00:27:43.514
before they move on.

00:27:43.514 --> 00:27:46.138
Jon: Is it also the case
that you're starting

00:27:46.138 --> 00:27:49.233
to get telemetry that's
telling you something

00:27:49.233 --> 00:27:50.960
about the videos, how
they're being used and

00:27:50.960 --> 00:27:54.136
what the before and after comparison is?

00:27:54.136 --> 00:27:56.707
Sal: The software
integration, I've done it

00:27:56.707 --> 00:27:59.237
pretty heavily with a
couple of summer camps

00:27:59.237 --> 00:28:03.380
out here in the Bay Area
and I haven't gotten

00:28:03.380 --> 00:28:05.549
to the point yet where
I'm testing two videos

00:28:05.549 --> 00:28:06.437
on the same concept.

00:28:06.437 --> 00:28:08.522
I don't have a lot of
multiple videos for the

00:28:08.522 --> 00:28:10.638
same concept yet, so I'm
tweaking on that level,

00:28:10.638 --> 00:28:15.215
but we have seen, we
did it with a group of

00:28:15.215 --> 00:28:17.627
rising 8th graders; these were low and

00:28:17.627 --> 00:28:19.962
middle income, mainly minority students,

00:28:19.962 --> 00:28:21.548
and this was a summer program to get them

00:28:21.548 --> 00:28:25.528
prepared for Algebra and pre-Algebra.

00:28:25.528 --> 00:28:26.878
We saw it. When we started the students at

00:28:26.878 --> 00:28:30.236
1 + 1 = 2, there were just
some subset of students

00:28:30.236 --> 00:28:33.237
who had just very, very
basic deficiencies;

00:28:33.237 --> 00:28:37.545
subtracting decimals or
multiplying negative numbers.

00:28:37.545 --> 00:28:40.138
You could see it in the
data, they spent a little

00:28:40.138 --> 00:28:42.051
more time on those concepts and everyone

00:28:42.051 --> 00:28:43.838
else in the class, everyone
else thought it was

00:28:43.838 --> 00:28:46.337
just a review, but these
students, it was a gap.

00:28:46.337 --> 00:28:47.637
Once they got through that gap,

00:28:47.637 --> 00:28:49.050
they raced forward and
they were just as good

00:28:49.050 --> 00:28:50.340
as anybody else.

00:28:50.340 --> 00:28:52.546
I think that speaks to
why there's a lot of ...

00:28:52.546 --> 00:28:54.043
If you go into an Algebra
class, and there's

00:28:54.043 --> 00:28:56.381
a lot of great teachers
with high potential

00:28:56.381 --> 00:28:58.737
students talking past each other because

00:28:58.737 --> 00:29:02.031
the teacher can't address
the arithmetic issue

00:29:02.031 --> 00:29:04.136
and the student may
not even know they have

00:29:04.136 --> 00:29:06.338
that deficiency, or if they
do, they're embarrassed

00:29:06.338 --> 00:29:08.382
to bring it up, so when
you allow people to

00:29:08.382 --> 00:29:11.237
do this at their own pace,
you do see it in the data

00:29:11.237 --> 00:29:14.437
that it's actually hard to predict who's

00:29:14.437 --> 00:29:16.543
going to end up at
different points because

00:29:16.543 --> 00:29:19.050
if you give someone the
time to fill in their gaps,

00:29:19.050 --> 00:29:20.629
you might have thought
that they were a slow

00:29:20.629 --> 00:29:22.111
student before, but
once that gap is filled,

00:29:22.111 --> 00:29:24.433
they actually progress quite fast.

00:29:24.433 --> 00:29:27.725
Jon: It seems on the one
hand, unimaginable that

00:29:27.725 --> 00:29:32.034
we could be here in the
21st century and not

00:29:32.034 --> 00:29:36.034
have widely-available
tools online already for

00:29:36.034 --> 00:29:37.818
people to do those kinds of things,

00:29:37.818 --> 00:29:39.534
to do those kinds of remediations,

00:29:39.534 --> 00:29:41.833
but on the other hand, I
guess it seemed kind of

00:29:41.833 --> 00:29:44.431
unimaginable that millions
of people would be

00:29:44.431 --> 00:29:45.933
driving past this freeway intersection ...

00:29:45.933 --> 00:29:47.833
(laughter)

00:29:47.833 --> 00:29:50.433
Nobody bothered to tell
them where North 5 was.

00:29:50.433 --> 00:29:57.218
Sal: The freeway intersection,
that one's actually,

00:29:57.218 --> 00:30:01.248
in the education context,
there's some obvious

00:30:01.248 --> 00:30:04.333
forces as to why that doesn't exist today.

00:30:04.333 --> 00:30:06.434
I think everyone who's trying to innovate

00:30:06.434 --> 00:30:08.634
in education is trying to do it within the

00:30:08.634 --> 00:30:10.661
current context so
they're trying to go into

00:30:10.661 --> 00:30:13.432
school systems and try
to incrementally change

00:30:13.432 --> 00:30:15.388
what teachers are doing
one way or the other.

00:30:15.388 --> 00:30:19.333
It's very hard to move
the beast, so to speak.

00:30:19.333 --> 00:30:21.432
Then you have all these people on the

00:30:21.432 --> 00:30:23.118
for-profit realm and I don't know,

00:30:23.118 --> 00:30:24.325
they're just trying to sell something for

00:30:24.325 --> 00:30:26.255
$19.95 a month and I
don't think they're ...

00:30:26.255 --> 00:30:27.830
They're thinking too deeply about how

00:30:27.830 --> 00:30:30.639
to do it right, regardless
of economic consequences.

00:30:30.639 --> 00:30:32.757
Jon: You're right. I think
it almost has to come

00:30:32.757 --> 00:30:35.665
from the outside and ...
I don't know if you know

00:30:35.665 --> 00:30:37.749
Graham Glass, he's a
friend of mine and he's

00:30:37.749 --> 00:30:41.633
working on service called EDU 2.0 ...

00:30:41.633 --> 00:30:42.846
Sal: EDU 2.0?

00:30:42.846 --> 00:30:46.671
Jon: You might be curious about that.

00:30:46.671 --> 00:30:49.000
He's coming at it partly from a courseware

00:30:49.000 --> 00:30:53.418
perspective, but also very
much from a project-based

00:30:53.418 --> 00:30:58.252
learning perspective
where the notion is ...

00:30:58.252 --> 00:30:59.844
It's kind of like this, right?

00:30:59.844 --> 00:31:02.740
I could have in my
school, I could have a kid

00:31:02.740 --> 00:31:04.839
who's interested in, I don't know,

00:31:04.839 --> 00:31:11.085
making and flying rockets, and there might

00:31:11.085 --> 00:31:13.252
or might not be somebody in the school who

00:31:13.252 --> 00:31:15.526
could mentor the kid.

00:31:15.526 --> 00:31:17.325
If there is, then you
have one of those lucky

00:31:17.325 --> 00:31:19.170
circumstances where the kid has a great

00:31:19.170 --> 00:31:23.841
experience and if not,
that capability goes

00:31:23.841 --> 00:31:27.753
untapped in the kid, but on the internet,

00:31:27.753 --> 00:31:29.664
we have the possibility to bring people

00:31:29.664 --> 00:31:31.761
together across time and space, right?

00:31:31.761 --> 00:31:34.841
So you have the possibility of bringing a

00:31:34.841 --> 00:31:39.250
mentor in from someplace
else and assembling

00:31:39.250 --> 00:31:43.826
mentors and students
in that way and letting

00:31:43.826 --> 00:31:46.225
them actually then pursue the stuff that

00:31:46.225 --> 00:31:48.941
excites them and do things, that's a nice

00:31:48.941 --> 00:31:52.337
complimentary model;
but again, he's coming

00:31:52.337 --> 00:31:56.762
completely from the
outside and I think that's

00:31:56.762 --> 00:31:59.441
probably where all the interesting action

00:31:59.441 --> 00:32:01.748
is going to be, it's going to have to be.

00:32:01.748 --> 00:32:05.254
Sal: I agree. I think
that's where it's all going.

00:32:05.254 --> 00:32:07.673
You have similar to Kahn
Academy has video content,

00:32:07.673 --> 00:32:08.940
it's going to have
software, it's going to do

00:32:08.940 --> 00:32:11.433
analytics on what's working
for different students

00:32:11.433 --> 00:32:12.581
and where the different
students are, but then

00:32:12.581 --> 00:32:15.173
you overlay that with
kind of a peer-to-peer

00:32:15.173 --> 00:32:17.940
interaction like you just described where

00:32:17.940 --> 00:32:20.584
the software knows that for Sal,

00:32:20.584 --> 00:32:22.435
the videos haven't been enough and even

00:32:22.435 --> 00:32:25.240
the software, he's struggling with it,

00:32:25.240 --> 00:32:30.587
I know his schedule and
I know a schedule of

00:32:30.587 --> 00:32:35.140
a young woman in Mexico City who's proven

00:32:35.140 --> 00:32:38.626
through the site that she
knows what she's doing

00:32:38.626 --> 00:32:39.921
and she's a high-regarded tutor,

00:32:39.921 --> 00:32:41.585
maybe she might be the same age as Sal or

00:32:41.585 --> 00:32:44.340
a little older or a practicing engineer or

00:32:44.340 --> 00:32:46.341
who knows what and you pair them up.

00:32:46.341 --> 00:32:49.257
You can even record the
interaction and then

00:32:49.257 --> 00:32:51.339
you capture the data,
the before and after,

00:32:51.339 --> 00:32:53.336
of Sal's performance and his ratings.

00:32:53.336 --> 00:32:55.757
Then all of a sudden, you have a very rich

00:32:55.757 --> 00:32:58.440
kind of peer-to-peer across time and space

00:32:58.440 --> 00:33:00.751
interactions amongst people all over.

00:33:00.751 --> 00:33:03.339
Obviously that has other
implications beyond

00:33:03.339 --> 00:33:05.430
just teaching people Math.

00:33:05.430 --> 00:33:07.639
It can do a lot in
terms of just awareness.

00:33:07.639 --> 00:33:10.632
Jon: Yeah. Another guy
I should mention here is

00:33:10.632 --> 00:33:13.339
Tim Fahlberg who's a Math teacher.

00:33:13.339 --> 00:33:16.740
He got interested in
making, well he actually

00:33:16.740 --> 00:33:19.490
has a coined term for
these things; Mathcasts.

00:33:19.490 --> 00:33:23.156
He's been doing them with
Camptasia for a long time.

00:33:23.156 --> 00:33:25.339
One of the great things
that he told me about

00:33:25.339 --> 00:33:27.133
a project where they were working with,

00:33:27.133 --> 00:33:29.984
I think it was elementary school kids

00:33:29.984 --> 00:33:34.441
in Australia, is that
he wound up having those

00:33:34.441 --> 00:33:39.838
kids make educational
videos for each other.

00:33:39.838 --> 00:33:43.244
They got to be the
teacher and in the process

00:33:43.244 --> 00:33:45.743
of teaching, as you do when teach,

00:33:45.743 --> 00:33:48.743
they also learned really well.

00:33:48.743 --> 00:33:51.245
They have a nice little
collection of these

00:33:51.245 --> 00:33:55.019
problem-solving videos made
by 4th graders in Perth

00:33:55.019 --> 00:33:56.975
or Sydney or somewhere.

00:33:56.975 --> 00:34:01.531
Sal: That's huge ...
on a massive scale and

00:34:01.531 --> 00:34:04.454
even if you have a thousand versions of

00:34:04.454 --> 00:34:06.420
adding negative numbers
from a thousand students,

00:34:06.420 --> 00:34:09.437
as you said, just them
going through the motions

00:34:09.437 --> 00:34:11.236
of doing it, they're
going to get a lot deeper

00:34:11.236 --> 00:34:13.738
understanding, but even better ...

00:34:13.738 --> 00:34:17.738
I like to review some of my own.

00:34:17.738 --> 00:34:18.637
What's better than notes?

00:34:18.637 --> 00:34:20.950
It's your own recording of how you thought

00:34:20.950 --> 00:34:22.326
about it two months ago
when you were really

00:34:22.326 --> 00:34:25.238
deeply engaged in it,
so you have that aspect

00:34:25.238 --> 00:34:26.638
and then you have the
aspect you might be able

00:34:26.638 --> 00:34:29.736
to identify the next great
teacher that way as well.

00:34:29.736 --> 00:34:32.454
Jon: Exactly. Exactly.

00:34:32.454 --> 00:34:34.538
Sal: And I'll throw this out ...

00:34:34.538 --> 00:34:36.538
I think great teachers were great when

00:34:36.538 --> 00:34:38.400
they were 13. (laughs) I don't think much

00:34:38.400 --> 00:34:40.042
happens between 13 and 30.

00:34:40.042 --> 00:34:44.736
Jon: I think that's true
and in the Bill Gates

00:34:44.736 --> 00:34:47.338
Ted Talk that I just watched,

00:34:47.338 --> 00:34:50.522
they said something similar which is that

00:34:50.522 --> 00:34:54.457
actually teachers in
their first couple years

00:34:54.457 --> 00:34:58.790
go through an experience
curve, but after that

00:34:58.790 --> 00:35:00.787
they're pretty much where
they're going to be.

00:35:00.787 --> 00:35:02.372
Sal: Right, and you know
in the traditional school

00:35:02.372 --> 00:35:04.031
system though, that experience curve

00:35:04.031 --> 00:35:05.435
is all around classroom management.

00:35:05.435 --> 00:35:06.539
I've talked to some of my friends who are

00:35:06.539 --> 00:35:07.373
teachers and they're like,

00:35:07.373 --> 00:35:10.042
"You don't have to do
this," and it's true,

00:35:10.042 --> 00:35:11.336
I don't have to do any of that.

00:35:11.336 --> 00:35:12.636
I think that's where a
lot of the experience

00:35:12.636 --> 00:35:18.338
curve is more than ...
You know, you're teaching

00:35:18.338 --> 00:35:20.237
30 or 25 kids so it's a
different concept than ...

00:35:20.237 --> 00:35:22.536
But I think they should be able to explain

00:35:22.536 --> 00:35:25.138
in a patient and empathetic way.

00:35:25.138 --> 00:35:26.368
What you know on YouTube,

00:35:26.368 --> 00:35:28.238
I think that's something,

00:35:28.238 --> 00:35:30.333
once you have a little bit of maturity,

00:35:30.333 --> 00:35:31.543
you'll probably be able to do as good as

00:35:31.543 --> 00:35:32.634
you'll ever be able to do it.

00:35:32.634 --> 00:35:38.039
Jon: Have you gotten
into areas where you are

00:35:38.039 --> 00:35:39.872
teaching yourself things that you hadn't

00:35:39.872 --> 00:35:42.208
previously known or known well?

00:35:42.208 --> 00:35:45.818
Sal: Oh yeah, tons of
... Frankly, that's the

00:35:45.818 --> 00:35:48.553
real fun for me is that I
get to revisit some things

00:35:48.553 --> 00:35:52.643
that I just did superficially
when I was in school

00:35:52.643 --> 00:35:55.539
and now I get to really
dive deep on my own

00:35:55.539 --> 00:35:57.453
timeframe without any pressure and really

00:35:57.453 --> 00:35:59.788
get to understand what those concepts are.

00:35:59.788 --> 00:36:01.137
The one example I was given was entropy.

00:36:01.137 --> 00:36:03.338
I wasn't a Chemistry jock in high school.

00:36:03.338 --> 00:36:05.436
I just did what I needed
to do to well in the class,

00:36:05.436 --> 00:36:07.637
but I didn't know it
intuitively the way I might have

00:36:07.637 --> 00:36:09.874
known Math or Computer Science,

00:36:09.874 --> 00:36:12.225
so that's been a great experience for me.

00:36:12.225 --> 00:36:14.822
I started in September
and I for two months,

00:36:14.822 --> 00:36:17.133
I really just became a Chemistry geek.

00:36:17.133 --> 00:36:18.737
I would call up my friends
who were professors

00:36:18.737 --> 00:36:20.437
at Stanford and MIT and say,

00:36:20.437 --> 00:36:23.375
"Hey, let's talk about entropy." (laughs)

00:36:23.375 --> 00:36:26.646
It's amazing, I think one
point of [unintelligible]

00:36:26.646 --> 00:36:29.325
I got from them is they envied me because

00:36:29.325 --> 00:36:31.840
when they were doing their PhDs,

00:36:31.840 --> 00:36:34.528
they couldn't go to their advisor and say,

00:36:34.528 --> 00:36:35.740
"Let's talk about entropy," because that's

00:36:35.740 --> 00:36:38.436
something that you were
supposed to have known

00:36:38.436 --> 00:36:40.840
when you took AP Chemistry or

00:36:40.840 --> 00:36:43.341
freshman level Chemistry,
but the reality is

00:36:43.341 --> 00:36:45.927
very few people do have
an intuition of say,

00:36:45.927 --> 00:36:50.539
entropy, including some professors today.

00:36:50.539 --> 00:36:53.820
Actually entropy examples,
once I was able to

00:36:53.820 --> 00:36:56.318
get the idea distilled in my mind,

00:36:56.318 --> 00:36:58.159
I realized that most
explanations out there

00:36:58.159 --> 00:37:01.238
were on some level, wrong.

00:37:01.238 --> 00:37:03.706
Jon: In what way?

00:37:03.706 --> 00:37:05.740
Sal: The way you learn
it in Chemistry class,

00:37:05.740 --> 00:37:08.539
the teacher says, "Entropy is disorder,"

00:37:08.539 --> 00:37:11.644
which isn't incorrect, but they say,

00:37:11.644 --> 00:37:13.787
"A clean room has low entropy and

00:37:13.787 --> 00:37:16.139
a dirty room has high entropy."

00:37:16.139 --> 00:37:18.142
So you go to the exam, they say,

00:37:18.142 --> 00:37:19.344
"This room is dirty, this one is clean.

00:37:19.344 --> 00:37:20.242
Which one has higher
entropy?" and you just

00:37:20.242 --> 00:37:22.238
kind of write it down.

00:37:22.238 --> 00:37:27.740
That's wrong. It's
missing the whole point of

00:37:27.740 --> 00:37:29.459
what entropy is. It took me a while to

00:37:29.459 --> 00:37:33.541
distill this idea that in Chemistry or

00:37:33.541 --> 00:37:36.045
Thermodynamics you have

00:37:36.045 --> 00:37:38.436
macrostates and microstates.

00:37:38.436 --> 00:37:40.378
Microstates are: you're
looking at a particular

00:37:40.378 --> 00:37:42.378
atom and you're looking
at its kinetic energy,

00:37:42.378 --> 00:37:43.342
you're looking at its momentum,

00:37:43.342 --> 00:37:45.433
you're looking at the
vector of its velocity,

00:37:45.433 --> 00:37:47.929
and if you looked at
the microstate of every

00:37:47.929 --> 00:37:49.532
little atom in a container,

00:37:49.532 --> 00:37:50.948
you could run a computer simulation and

00:37:50.948 --> 00:37:52.572
maybe might figure out what might happen

00:37:52.572 --> 00:37:55.813
the very next moment ...
Those are microstates.

00:37:55.813 --> 00:37:58.066
Macrostates are: you look
at the whole container

00:37:58.066 --> 00:38:01.748
and you describe all of the particles,

00:38:01.748 --> 00:38:03.980
almost you can think of
statistics for all the

00:38:03.980 --> 00:38:06.405
particles, so temperature is a macrostate;

00:38:06.405 --> 00:38:08.449
it's the average kinetic energy for all of

00:38:08.449 --> 00:38:11.350
the particles; so entropy similarly,

00:38:11.350 --> 00:38:14.853
is a macrostate, but it's
almost a metamacrostate.

00:38:14.853 --> 00:38:19.481
It's this idea of not what
state are the particles in,

00:38:19.481 --> 00:38:23.071
it's how many potential
states can the particles have.

00:38:23.071 --> 00:38:26.152
In the messy clean room analogy,

00:38:26.152 --> 00:38:27.566
the reason why I say
that's wrong is because

00:38:27.566 --> 00:38:29.749
each of those are particular states.

00:38:29.749 --> 00:38:32.845
It doesn't say, "A messy
room," it depends on

00:38:32.845 --> 00:38:34.530
what level you're
thinking about as states,

00:38:34.530 --> 00:38:36.946
but it has the same number, because of its

00:38:36.946 --> 00:38:39.429
temperature, its atoms
have the same number

00:38:39.429 --> 00:38:43.237
of potential states as the clean room.

00:38:43.237 --> 00:38:46.360
I guess I'll give you a
really hot, clean room

00:38:46.360 --> 00:38:48.450
with things flying around in the air,

00:38:48.450 --> 00:38:51.946
is going to be higher entropy than

00:38:51.946 --> 00:38:53.734
a cold, dirty room with different things

00:38:53.734 --> 00:38:54.948
(laughs) are flying around in the air.

00:38:54.948 --> 00:38:56.444
Jon: (laughs)

00:38:56.444 --> 00:38:58.777
Sal: It's the number of
states that something

00:38:58.777 --> 00:39:00.700
can have and you can
even translate that into

00:39:00.700 --> 00:39:03.046
computer science. It's
the amount of information

00:39:03.046 --> 00:39:07.039
you need to describe
the state of a system.

00:39:07.039 --> 00:39:08.745
So if something that can have more states,

00:39:08.745 --> 00:39:11.365
you need to have more bits to describe its

00:39:11.365 --> 00:39:14.691
particular state ... And
that's completely lost,

00:39:14.691 --> 00:39:17.134
at least it was in the
Chemistry classes I took.

00:39:17.134 --> 00:39:20.423
Jon: That's good. I bet
one of the other areas

00:39:20.423 --> 00:39:23.236
that's going to be very
fruitful for you in

00:39:23.236 --> 00:39:27.476
coming years is Energy Literacy?

00:39:27.476 --> 00:39:29.980
Sal: These are questions ...

00:39:29.980 --> 00:39:32.836
A lot of people request
a lot of clean tech

00:39:32.836 --> 00:39:34.644
lectures, a lot of
people want me to settle

00:39:34.644 --> 00:39:37.555
the global warming debate,
the health care debate,

00:39:37.555 --> 00:39:40.434
these are things I look
forward to eventually.

00:39:40.434 --> 00:39:41.436
I don't know if I'm
going to be able to catch

00:39:41.436 --> 00:39:45.235
up with the kind of new cycle, but I think

00:39:45.235 --> 00:39:46.940
there is that opportunity
where I can dig deep

00:39:46.940 --> 00:39:50.394
into things and as
objectively as possible.

00:39:50.394 --> 00:39:51.735
I think people trust
me because they learned

00:39:51.735 --> 00:39:55.061
Calculus from me, that
I don't have a political

00:39:55.061 --> 00:39:57.973
kind of ax to grind and I can give them a

00:39:57.973 --> 00:40:01.640
fairly objective, hopefully
objective take on things.

00:40:01.640 --> 00:40:03.435
Jon: When I say energy literacy,

00:40:03.435 --> 00:40:05.234
I'm talking about, to give you an example,

00:40:05.234 --> 00:40:06.834
I was looking at some of
your stuff the other day

00:40:06.834 --> 00:40:09.020
and I've also been,
recently I've been hanging

00:40:09.020 --> 00:40:12.650
around on Wolfram Alpha a lot because it's

00:40:12.650 --> 00:40:18.234
a really helpful way of making conversions

00:40:18.234 --> 00:40:22.341
between units and
different forms of energy

00:40:22.341 --> 00:40:24.920
and matter and so on, and understanding

00:40:24.920 --> 00:40:29.840
like I don't know, I
put in my electric bill

00:40:29.840 --> 00:40:31.742
was such-and-such kilowatt hours,

00:40:31.742 --> 00:40:33.657
it was like 1,200 kilowatt
hours, right, like

00:40:33.657 --> 00:40:35.718
"What is that? What
does that really mean?"

00:40:35.718 --> 00:40:38.018
It'll give it to you in terms of gasoline.

00:40:38.018 --> 00:40:38.802
Sal: That's cool.

00:40:38.802 --> 00:40:42.846
Jon: As a society, we are broadly speaking

00:40:42.846 --> 00:40:45.938
feeling like we need to
have a better handle on,

00:40:45.938 --> 00:40:49.350
"What does this stuff
really mean?" in terms that

00:40:49.350 --> 00:40:51.367
we can freely interconvert, and

00:40:51.367 --> 00:40:53.138
we're not so good at that.

00:40:53.138 --> 00:40:55.268
I went from one of your examples back to

00:40:55.268 --> 00:40:58.026
Wolfram Alpha and was thinking,

00:40:58.026 --> 00:41:01.677
I guess it was just the elevator that

00:41:01.677 --> 00:41:05.539
weighs so much and is
100 meters off the ground

00:41:05.539 --> 00:41:08.410
and it's got this much
potential energy and

00:41:08.410 --> 00:41:10.640
this much kinetic energy
when it hits the floor,

00:41:10.640 --> 00:41:13.837
and I looked at that relative
to gallons of gasoline

00:41:13.837 --> 00:41:16.138
and kilowatt hours of my electric bill and

00:41:16.138 --> 00:41:18.683
it was like wow, I
wouldn't have thought this,

00:41:18.683 --> 00:41:23.263
but apparently the energy
I burn as electricity

00:41:23.263 --> 00:41:26.767
in my home in a day could take an elevator

00:41:26.767 --> 00:41:28.440
to the top of the Empire State Building

00:41:28.440 --> 00:41:30.339
10 or 20 times.

00:41:30.339 --> 00:41:32.039
I wouldn't have thought that.

00:41:32.039 --> 00:41:35.741
Sal: That's something and it gives people

00:41:35.741 --> 00:41:37.848
a context for what a kilowatt hour is,

00:41:37.848 --> 00:41:44.763
what a calorie is, any of the above.

00:41:44.763 --> 00:41:47.692
Jon: Yeah, because we really
lack those intuitions.

00:41:47.692 --> 00:41:49.038
It kind of just struck me because I wrote

00:41:49.038 --> 00:41:53.038
about this on my blog and
I realized all my life

00:41:53.038 --> 00:41:57.269
I've thought about
electric bills as dollars,

00:41:57.269 --> 00:41:59.937
and I mentioned this was my electric bill,

00:41:59.937 --> 00:42:01.346
it was this many kilowatt hours and people

00:42:01.346 --> 00:42:04.185
wrote in from Belgium and the Netherlands

00:42:04.185 --> 00:42:06.426
and they said, "Really?"

00:42:06.426 --> 00:42:09.437
What made me surprised was that

00:42:09.437 --> 00:42:10.684
I suddenly realized, you know what?

00:42:10.684 --> 00:42:12.639
They have an intuition about what it means

00:42:12.639 --> 00:42:15.323
to be a quantity of kilowatt hours.

00:42:15.323 --> 00:42:20.912
The first order for them
isn't dollars or Euros,

00:42:20.912 --> 00:42:21.950
it's kilowatt hours.

00:42:21.950 --> 00:42:23.548
That actually means something to them

00:42:23.548 --> 00:42:24.872
in a way that it doesn't to us.

00:42:24.872 --> 00:42:26.166
I don't know, I thought that was ...

00:42:26.166 --> 00:42:28.248
Sal: You're inspiring me to do at least

00:42:28.248 --> 00:42:31.644
a couple of videos on it.

00:42:31.644 --> 00:42:34.845
Jon: Well listen, I think I should let you

00:42:34.845 --> 00:42:37.499
get back to this stuff that you do.

00:42:37.499 --> 00:42:39.497
I really don't have a lot else to ask you,

00:42:39.497 --> 00:42:42.747
but I just want to thank you for what

00:42:42.747 --> 00:42:46.075
you're doing, I think it's brilliant and

00:42:46.075 --> 00:42:48.166
I am happy to see it happening.

00:42:48.166 --> 00:42:51.646
I'm happy for you to be excited and

00:42:51.646 --> 00:42:52.847
enjoying doing this.

00:42:52.847 --> 00:42:55.448
I know that you're going to find a way to

00:42:55.448 --> 00:42:58.347
make this work out as a profession,

00:42:58.347 --> 00:43:02.351
as a career, as a lifetime mission.

00:43:02.351 --> 00:43:03.237
Sal: Great, I appreciate it.

00:43:03.237 --> 00:43:04.448
I have that gut feeling.

00:43:04.448 --> 00:43:06.449
It's being done as a
non-profit, but we need

00:43:06.449 --> 00:43:08.668
to get to the point the not-for-profit can

00:43:08.668 --> 00:43:10.849
pay my rent.

00:43:10.849 --> 00:43:13.947
Jon: In my mind, there's
no way that can't happen.

00:43:13.947 --> 00:43:15.584
Sal: Yeah, I'm getting that gut feeling.

00:43:15.584 --> 00:43:17.644
Hopefully you'll hear
some interesting news

00:43:17.644 --> 00:43:22.162
from us ... I don't want
to jinx us, but we'll see.

00:43:22.162 --> 00:43:23.575
Jon: Okay. Thanks a lot, Sal.

00:43:23.575 --> 00:43:25.347
Sal: Thanks a lot, Jon.

00:43:25.347 --> 00:43:36.164
(jazz music)

00:43:36.164 --> 00:43:37.447
Voiceover: You've been listening to

00:43:37.447 --> 00:43:39.940
"Interviews With Innovators"
with host Jon Udell.

00:43:39.940 --> 00:43:41.740
Our audio files are delivered by

00:43:41.740 --> 00:43:43.540
Limelight Networks, the high-performance

00:43:43.540 --> 00:43:46.041
content delivery network
for digital media.

00:43:46.041 --> 00:43:49.669
Conversations Network is
a 501(c)(3) non-profit

00:43:49.669 --> 00:43:50.931
and we need your help.

00:43:50.931 --> 00:43:53.251
For a tax-deductible
donation of as little as

00:43:53.251 --> 00:43:56.135
$5 per month, you can support this channel

00:43:56.135 --> 00:43:58.509
and the rest of the Conversations Network.

00:43:58.509 --> 00:44:00.941
Please visit conversationsnetwork.org

00:44:00.941 --> 00:44:02.995
to become a member and help us continue to

00:44:02.995 --> 00:44:05.341
bring our programs to the world for free.

00:44:05.341 --> 00:44:06.941
The post-production
audio engineer for this

00:44:06.941 --> 00:44:09.409
program was Jon Udell.

00:44:09.409 --> 00:44:12.440
Our website editor was Joel Tscherne.

00:44:12.440 --> 00:44:15.441
The series producer is Joel Tscherne.

00:44:15.441 --> 00:44:16.539
This is Phil Windley.

00:44:16.539 --> 00:44:18.340
I hope you'll join me
next time for another

00:44:18.340 --> 00:44:20.406
great edition of
"Interviews With Innovators"

00:44:20.406 --> 00:44:25.039
on IT Conversations.


Sampling distribution of the sample mean

WEBVTT
Kind: captions
Language: en

00:00:00.374 --> 00:00:02.040
In the last video,
we learned about what

00:00:02.040 --> 00:00:05.189
is quite possibly the most
profound idea in statistics,

00:00:05.189 --> 00:00:06.730
and that's the
central limit theorem.

00:00:06.730 --> 00:00:07.730
And the reason why
it's so neat is,

00:00:07.730 --> 00:00:09.740
we could start with
any distribution that

00:00:09.740 --> 00:00:13.152
has a well defined mean
and variance-- actually,

00:00:13.152 --> 00:00:14.610
I wrote the standard
deviation here

00:00:14.610 --> 00:00:16.760
in the last video, that
should be the mean,

00:00:16.760 --> 00:00:18.330
and let's say it
has some variance.

00:00:18.330 --> 00:00:19.990
I could write it
like that, or I could

00:00:19.990 --> 00:00:21.960
write the standard
deviation there.

00:00:21.960 --> 00:00:23.480
But as long as it
has a well defined

00:00:23.480 --> 00:00:25.240
mean and standard
deviation, I don't

00:00:25.240 --> 00:00:27.890
care what the
distribution looks like.

00:00:27.890 --> 00:00:30.730
What I can do is take samples--
in the last video of say,

00:00:30.730 --> 00:00:35.060
size four-- that means
I take literally four

00:00:35.060 --> 00:00:38.120
instances of this random
variable, this is one example.

00:00:38.120 --> 00:00:40.480
I take their mean,
and I consider

00:00:40.480 --> 00:00:42.880
this the sample mean
from my first trial,

00:00:42.880 --> 00:00:44.870
or you could almost say
for my first sample.

00:00:44.870 --> 00:00:47.080
I know it's very confusing,
because you can consider

00:00:47.080 --> 00:00:49.000
that a sample, the
set to be a sample,

00:00:49.000 --> 00:00:52.787
or you could consider each
member of the set is a sample.

00:00:52.787 --> 00:00:54.620
So that can be a little
bit confusing there.

00:00:54.620 --> 00:00:56.050
But I have this
first sample mean,

00:00:56.050 --> 00:00:57.758
and then I keep doing
that over and over.

00:00:57.758 --> 00:01:00.430
In my second sample,
my sample size is four.

00:01:00.430 --> 00:01:02.970
I got four instances of
this random variable,

00:01:02.970 --> 00:01:05.770
I average them, I have
another sample mean.

00:01:05.770 --> 00:01:08.720
And the cool thing about
the central limit theorem,

00:01:08.720 --> 00:01:10.800
is as I keep plotting
the frequency

00:01:10.800 --> 00:01:13.100
distribution of my
sample means, it

00:01:13.100 --> 00:01:14.730
starts to approach
something that

00:01:14.730 --> 00:01:17.150
approximates the
normal distribution.

00:01:17.150 --> 00:01:19.310
And it's going to do a
better job of approximating

00:01:19.310 --> 00:01:22.614
that normal distribution
as n gets larger.

00:01:22.614 --> 00:01:24.280
And just so we have
a little terminology

00:01:24.280 --> 00:01:28.030
on our belt, this
frequency distribution

00:01:28.030 --> 00:01:30.640
right here that
I've plotted out,

00:01:30.640 --> 00:01:34.900
or here, or up here that
I started plotting out,

00:01:34.900 --> 00:01:37.520
that is called-- and
it's kind of confusing,

00:01:37.520 --> 00:01:39.590
because we use the word
sample so much-- that

00:01:39.590 --> 00:01:51.100
is called the sampling
distribution of the sample

00:01:51.100 --> 00:01:52.191
mean.

00:01:52.191 --> 00:01:53.690
And let's dissect
this a little bit,

00:01:53.690 --> 00:01:55.670
just so that this
long description

00:01:55.670 --> 00:01:58.810
of this distribution starts
to make a little bit of sense.

00:01:58.810 --> 00:02:01.230
When we say it's the
sampling distribution,

00:02:01.230 --> 00:02:03.060
that's telling us that
it's being derived

00:02:03.060 --> 00:02:06.300
from-- it's a distribution
of some statistic, which

00:02:06.300 --> 00:02:08.550
in this case happens
to be the sample mean--

00:02:08.550 --> 00:02:10.870
and we're deriving
it from samples

00:02:10.870 --> 00:02:12.190
of an original distribution.

00:02:12.190 --> 00:02:13.790
So each of these.

00:02:13.790 --> 00:02:16.610
So this is my first sample,
my sample size is four.

00:02:16.610 --> 00:02:18.590
I'm using the
statistic, the mean.

00:02:18.590 --> 00:02:20.590
I actually could have
done it with other things,

00:02:20.590 --> 00:02:24.050
I could have done the mode or
the range or other statistics.

00:02:24.050 --> 00:02:28.460
But sampling distribution
of the sample mean

00:02:28.460 --> 00:02:29.710
is the most common one.

00:02:29.710 --> 00:02:31.690
It's probably, in
my mind, the best

00:02:31.690 --> 00:02:34.880
place to start learning about
the central limit theorem,

00:02:34.880 --> 00:02:37.160
and even frankly,
sampling distribution.

00:02:37.160 --> 00:02:38.320
So that's what it's called.

00:02:38.320 --> 00:02:39.720
And just as a little bit
of background-- and I'll

00:02:39.720 --> 00:02:41.230
prove this to you
experimentally,

00:02:41.230 --> 00:02:44.070
not mathematically, but
I think the experimental

00:02:44.070 --> 00:02:46.800
is on some levels more
satisfying with statistics--

00:02:46.800 --> 00:02:52.290
that this will
have the same mean

00:02:52.290 --> 00:02:54.630
as your original distribution.

00:02:54.630 --> 00:02:56.360
As your original
distribution right here.

00:02:56.360 --> 00:02:59.050
So it has the same mean, but
we'll see in the next video

00:02:59.050 --> 00:03:01.840
that this is actually going
to start approximating

00:03:01.840 --> 00:03:04.480
a normal distribution, even
though my original distribution

00:03:04.480 --> 00:03:07.510
that this is kind of generated
from, is completely non-normal.

00:03:07.510 --> 00:03:10.750
So let's do that with
this app right here.

00:03:10.750 --> 00:03:13.176
And just to give proper
credit where credit is due,

00:03:13.176 --> 00:03:15.550
this is-- I think was developed
at Rice University-- this

00:03:15.550 --> 00:03:16.675
is from onlinestatbook.com.

00:03:23.030 --> 00:03:25.280
This is their app, which I
think is a really neat app,

00:03:25.280 --> 00:03:28.040
because it really helps you
to visualize what a sampling

00:03:28.040 --> 00:03:30.330
distribution of
the sample mean is.

00:03:30.330 --> 00:03:33.680
So I can literally create my
own custom distribution here.

00:03:33.680 --> 00:03:35.900
So let me make
something kind of crazy.

00:03:35.900 --> 00:03:38.680
So you could do this, in
theory, with a discrete

00:03:38.680 --> 00:03:42.820
or a continuous probability
density function.

00:03:42.820 --> 00:03:45.370
But what they have here, we
could take on one of 32 values,

00:03:45.370 --> 00:03:47.578
and I'm just going to set
the different probabilities

00:03:47.578 --> 00:03:49.730
of getting any of
those 32 values.

00:03:49.730 --> 00:03:52.295
So clearly, this right here
is not a normal distribution.

00:03:52.295 --> 00:03:55.772
It looks a little bit bimodal,
but it doesn't have long tails.

00:03:55.772 --> 00:03:57.980
But what I want to do is,
first just use a simulation

00:03:57.980 --> 00:04:00.400
to understand, or to
better understand,

00:04:00.400 --> 00:04:02.840
what the sampling
distribution is all about.

00:04:02.840 --> 00:04:04.590
So what I'm going
to do is, I'm going

00:04:04.590 --> 00:04:07.010
to take-- we'll start
with-- five at a time.

00:04:07.010 --> 00:04:09.924
So my sample size
is going to be five.

00:04:09.924 --> 00:04:12.090
And so when I click animated,
what it's going to do,

00:04:12.090 --> 00:04:15.350
is it's going to take five
samples from this probability

00:04:15.350 --> 00:04:16.319
distribution function.

00:04:16.319 --> 00:04:18.110
It's going to take five
samples, and you're

00:04:18.110 --> 00:04:19.850
going to see them
when I click animated,

00:04:19.850 --> 00:04:22.350
it's going to average them and
plot the average down here.

00:04:22.350 --> 00:04:23.250
And then I'm going
to click it again,

00:04:23.250 --> 00:04:24.520
and it's going to do it again.

00:04:24.520 --> 00:04:26.520
So there you go, it got
five samples from there,

00:04:26.520 --> 00:04:29.440
it averaged them,
and it hit there.

00:04:29.440 --> 00:04:31.070
So what I just do?

00:04:31.070 --> 00:04:34.250
I clicked-- oh, I
wanted to clear that.

00:04:34.250 --> 00:04:36.042
Let me make this
bottom one none.

00:04:36.042 --> 00:04:37.250
So let me do that over again.

00:04:37.250 --> 00:04:39.780
So I'm going to
take five at time.

00:04:39.780 --> 00:04:43.210
So I took five samples from up
here, and then it took its mean

00:04:43.210 --> 00:04:44.390
and plotted the mean there.

00:04:44.390 --> 00:04:45.970
Let me do it again.

00:04:45.970 --> 00:04:48.390
Five samples from this
probability distribution

00:04:48.390 --> 00:04:50.110
function, plotted
it right there.

00:04:50.110 --> 00:04:51.080
I could keep doing it.

00:04:51.080 --> 00:04:52.390
It'll take some time.

00:04:52.390 --> 00:04:54.511
But you can see I
plotted it right there.

00:04:54.511 --> 00:04:57.010
Now I could do this 1,000 times,
it's going to take forever.

00:04:57.010 --> 00:04:59.370
Let's say I just wanted
to do it 1,000 times.

00:04:59.370 --> 00:05:01.080
So this program,
just to be clear,

00:05:01.080 --> 00:05:02.913
it's actually generating
the random numbers.

00:05:02.913 --> 00:05:04.500
This isn't like
a rigged program.

00:05:04.500 --> 00:05:07.000
It's actually going to generate
the random numbers according

00:05:07.000 --> 00:05:08.870
to this probability
distribution function.

00:05:08.870 --> 00:05:11.220
It's going to take five at
a time, find their means,

00:05:11.220 --> 00:05:12.110
and plot the means.

00:05:12.110 --> 00:05:15.020
So if I click 10,000, it's
going to do that 10,000 times.

00:05:15.020 --> 00:05:17.720
So it's going to take five
numbers from here 10,000 times

00:05:17.720 --> 00:05:21.090
and find their
means 10,000 times

00:05:21.090 --> 00:05:22.780
and then plot the
10,000 means here.

00:05:22.780 --> 00:05:25.144
So let's do that.

00:05:25.144 --> 00:05:25.810
So there you go.

00:05:25.810 --> 00:05:27.670
And notice it's
already looking a lot

00:05:27.670 --> 00:05:29.280
like a normal distribution.

00:05:29.280 --> 00:05:32.600
And like I said, the original
mean of my crazy distribution

00:05:32.600 --> 00:05:38.160
here was 14.45, and after doing
10,000 samples-- or 10,000

00:05:38.160 --> 00:05:41.040
trials-- my mean here is 14.42.

00:05:41.040 --> 00:05:43.492
So I'm already getting pretty
close to the mean there.

00:05:43.492 --> 00:05:45.950
My standard deviation, you
might notice, is less than that.

00:05:45.950 --> 00:05:47.990
We'll talk about that
in a future video.

00:05:47.990 --> 00:05:51.640
And the skew and
kurtosis, these are

00:05:51.640 --> 00:05:54.617
things that help us measure
how normal a distribution is.

00:05:54.617 --> 00:05:56.700
And I've talked a little
bit about it in the past,

00:05:56.700 --> 00:06:01.210
and let me actually just diverge
a little bit, it's interesting.

00:06:01.210 --> 00:06:03.190
And they're fairly
straightforward concepts.

00:06:03.190 --> 00:06:05.770
Skew literally
tells-- so if this

00:06:05.770 --> 00:06:08.740
is-- let me do it in
a different color--

00:06:08.740 --> 00:06:11.260
if this is a perfect
normal distribution--

00:06:11.260 --> 00:06:14.840
and clearly my drawing is
very far from perfect--

00:06:14.840 --> 00:06:17.600
if that's a perfect
distribution,

00:06:17.600 --> 00:06:19.556
this would have a skew of zero.

00:06:19.556 --> 00:06:20.930
If you have a
positive skew, that

00:06:20.930 --> 00:06:24.040
means you have a larger right
tail than you would otherwise

00:06:24.040 --> 00:06:24.540
expect.

00:06:24.540 --> 00:06:27.690
So something with a positive
skew might look like this.

00:06:27.690 --> 00:06:29.960
It would have a large
tail to the right.

00:06:29.960 --> 00:06:33.796
So this would be
a positive skew,

00:06:33.796 --> 00:06:35.420
which makes it a
little less than ideal

00:06:35.420 --> 00:06:36.472
for normal distribution.

00:06:36.472 --> 00:06:38.180
And a negative skew
would look like this,

00:06:38.180 --> 00:06:40.630
it has a long tail to the left.

00:06:40.630 --> 00:06:43.380
So negative skew
might look like that.

00:06:43.380 --> 00:06:44.842
So that is a negative skew.

00:06:44.842 --> 00:06:46.300
If you have trouble
remembering it,

00:06:46.300 --> 00:06:47.960
just remember which
direction the tail is going.

00:06:47.960 --> 00:06:49.990
This tail is going towards
a negative direction,

00:06:49.990 --> 00:06:51.930
this tail is going to
the positive direction.

00:06:51.930 --> 00:06:53.305
So if something
has no skew, that

00:06:53.305 --> 00:06:56.107
means that it's nice and
symmetrical around its mean.

00:06:56.107 --> 00:06:58.190
Now kurtosis, which sounds
like a very fancy word,

00:06:58.190 --> 00:07:03.720
is similarly not that
fancy of an idea.

00:07:03.720 --> 00:07:08.344
So once again, if I were to draw
a perfect normal distribution.

00:07:08.344 --> 00:07:10.260
Remember, there is no
one normal distribution,

00:07:10.260 --> 00:07:11.676
you could have
different means and

00:07:11.676 --> 00:07:14.600
different standard deviations.

00:07:14.600 --> 00:07:16.990
Let's say that's a perfect
normal distribution.

00:07:16.990 --> 00:07:21.820
If I have positive kurtosis,
what's going to happen

00:07:21.820 --> 00:07:24.160
is, I'm going to have
fatter tails-- let

00:07:24.160 --> 00:07:27.030
me draw it a little nicer than
that-- I'm going to have fatter

00:07:27.030 --> 00:07:30.729
tails, but I'm going to
have a more pointy peak.

00:07:30.729 --> 00:07:32.270
I didn't have to
draw it that pointy,

00:07:32.270 --> 00:07:33.440
let me draw it like this.

00:07:33.440 --> 00:07:36.250
I'm going to have
fatter tails, and I'm

00:07:36.250 --> 00:07:39.500
going to have a more pointy
peak than a normal distribution.

00:07:39.500 --> 00:07:41.820
So this right here
is positive kurtosis.

00:07:41.820 --> 00:07:44.572
So something that has
positive kurtosis-- depending

00:07:44.572 --> 00:07:46.280
on how positive it
is-- it tells you it's

00:07:46.280 --> 00:07:51.360
a little bit more pointy than
a real normal distribution.

00:07:51.360 --> 00:07:54.900
And negative kurtosis
has smaller tails,

00:07:54.900 --> 00:07:57.400
but it's smoother
near the middle.

00:07:57.400 --> 00:07:58.650
So it's like this.

00:07:58.650 --> 00:08:02.380
So something like this would
have negative kurtosis.

00:08:04.990 --> 00:08:07.740
And maybe in future videos we'll
explore that in more detail,

00:08:07.740 --> 00:08:09.420
but in the context
of the simulation,

00:08:09.420 --> 00:08:12.410
it's just telling us how
normal this distribution is.

00:08:12.410 --> 00:08:14.650
So when our sample
size was n equal 5

00:08:14.650 --> 00:08:16.590
and we did 10,000 trials,
we got pretty close

00:08:16.590 --> 00:08:17.820
to a normal distribution.

00:08:17.820 --> 00:08:21.304
Let's do another 10,000 trials,
just to see what happens.

00:08:21.304 --> 00:08:23.220
It looks even more like
a normal distribution.

00:08:23.220 --> 00:08:24.826
Our mean is now the
exact same number,

00:08:24.826 --> 00:08:26.450
but we still have a
little bit of skew,

00:08:26.450 --> 00:08:27.784
and a little bit of kurtosis.

00:08:27.784 --> 00:08:30.450
Now let's see what happens if we
do the same thing with a larger

00:08:30.450 --> 00:08:31.790
sample size.

00:08:31.790 --> 00:08:33.679
And we could actually
do them simultaneously.

00:08:33.679 --> 00:08:35.100
So here's n equal 5.

00:08:35.100 --> 00:08:37.909
Let's do here, n equals 25.

00:08:37.909 --> 00:08:39.380
Just let me clear them.

00:08:39.380 --> 00:08:42.179
I'm going to do the sampling
distribution of the sample

00:08:42.179 --> 00:08:42.932
mean.

00:08:42.932 --> 00:08:44.640
And I'm going to run
10,000 trials-- I'll

00:08:44.640 --> 00:08:47.367
do one animated trial, just so
you remember what's going on.

00:08:47.367 --> 00:08:49.700
So I'm literally taking first
five samples from up here,

00:08:49.700 --> 00:08:50.690
find their mean.

00:08:50.690 --> 00:08:55.420
Now I'm taking 25 samples
from up here, find its mean,

00:08:55.420 --> 00:08:56.990
and then plotting it down here.

00:08:56.990 --> 00:08:59.200
So here the sample size
is 25, here it's five.

00:08:59.200 --> 00:09:01.220
I'll do it one more time.

00:09:01.220 --> 00:09:03.310
I take five, get
the mean, plot it.

00:09:03.310 --> 00:09:07.960
Take 25, get the mean, and
then plot it down there.

00:09:07.960 --> 00:09:09.950
This is a larger sample size.

00:09:09.950 --> 00:09:15.660
Now that thing that I just did,
I'm going to do 10,000 times.

00:09:15.660 --> 00:09:17.390
And remember, our
first distribution

00:09:17.390 --> 00:09:21.150
was just this really crazy
very non-normal distribution,

00:09:21.150 --> 00:09:23.290
but once we did it--
whoops, I didn't

00:09:23.290 --> 00:09:24.340
want to make it that big.

00:09:27.380 --> 00:09:28.800
Scroll up a little bit.

00:09:28.800 --> 00:09:29.970
So here, what's interesting?

00:09:29.970 --> 00:09:31.630
I mean they both
look a little normal,

00:09:31.630 --> 00:09:33.505
but if you look at the
skew and the kurtosis,

00:09:33.505 --> 00:09:36.460
when our sample size is
larger, it's more normal.

00:09:36.460 --> 00:09:40.090
This has a lower skew than when
our sample size was only five.

00:09:40.090 --> 00:09:42.900
And it has a less
negative kurtosis

00:09:42.900 --> 00:09:44.750
than when our sample
size was five.

00:09:44.750 --> 00:09:47.179
So this is a more
normal distribution.

00:09:47.179 --> 00:09:49.220
And one thing that we're
going to explore further

00:09:49.220 --> 00:09:53.180
in a future video, is not only
is it more normal in its shape,

00:09:53.180 --> 00:09:55.810
but it's also tighter
fit around the mean.

00:09:55.810 --> 00:09:58.550
And you can even think about
why that kind of makes sense.

00:09:58.550 --> 00:10:01.370
When your sample size
is larger, your odds

00:10:01.370 --> 00:10:03.760
of getting really far away
from the mean is lower.

00:10:03.760 --> 00:10:05.200
Because it's very
low likelihood,

00:10:05.200 --> 00:10:07.249
if you're taking 25 samples,
or 100 samples, that

00:10:07.249 --> 00:10:09.540
you're just going to get a
bunch of stuff way out here,

00:10:09.540 --> 00:10:10.500
or a bunch of
stuff way out here.

00:10:10.500 --> 00:10:12.937
You're very likely to get a
reasonable spread of things.

00:10:12.937 --> 00:10:15.270
So it makes sense that your
mean-- your sample mean-- is

00:10:15.270 --> 00:10:17.664
less likely to be far
away from the mean.

00:10:17.664 --> 00:10:20.080
We're going to talk a little
bit more about in the future.

00:10:20.080 --> 00:10:21.940
But hopefully this
kind of satisfies you

00:10:21.940 --> 00:10:23.340
that-- at least
experimentally, I

00:10:23.340 --> 00:10:25.750
haven't proven it to you with
mathematical rigor, which

00:10:25.750 --> 00:10:27.370
hopefully we'll
do in the future.

00:10:27.370 --> 00:10:28.720
But hopefully this
satisfies you, at least

00:10:28.720 --> 00:10:30.928
experimentally, that the
central limit theorem really

00:10:30.928 --> 00:10:32.340
does apply to any distribution.

00:10:32.340 --> 00:10:34.210
I mean, this is a
crazy distribution.

00:10:34.210 --> 00:10:38.150
And I encourage you to use this
applet at onlinestatbook.com

00:10:38.150 --> 00:10:40.340
and experiment with
other crazy distributions

00:10:40.340 --> 00:10:41.540
to believe it for yourself.

00:10:41.540 --> 00:10:43.710
But the interesting
things are that we're

00:10:43.710 --> 00:10:45.140
approaching a
normal distribution,

00:10:45.140 --> 00:10:47.680
but as my sample
size got larger,

00:10:47.680 --> 00:10:51.143
it's a better fit for
a normal distribution.