This episode id sponsored by Brilliant. You’re faced with a dilemma.
There’s a rogue trolley headed straight
for 5 people.
There’s a lever which, when pulled, changes
the trolley’s course saving the 5 people,
but killing 1.
If you do nothing, 5 people will die.
If you pull the lever, you save 5 but kill
1 who would have otherwise been safe.
With only seconds to act, what do you do?
This is a famous question in philosophy called
The Trolley Problem and philosophers have
been arguing about it for centuries.
Some say the right thing to do is to pull the
lever, as saving 5 lives is obviously better
than saving 1.
Others argue that the act of pulling the lever
means that you are responsible for the death,
whereas inaction would just have been letting
fate happen.
We’re far from reaching a consensus on the
matter and the battle of right and wrong still
continues.
And that’s been fine… until now.
Self driving cars are estimated to be on roads
in the not too distant future and accidents will inevitably happen. Programmers
will need to make these decisions ahead of
time if the car is to act correctly in the
face of death.
So should we just round up all the ethicists
and philosophers and lock them in a room until
they can all agree?
Well a professor of computational social choice,
Ariel Procaccia from Carnegie Mellon University,
had a better idea.
The story starts with a visit to MIT where,
when talking to other computer scientists,
Prof. Procaccia and his PhD Ritesh heard about the most fascinating
experiment.
The Moral Machine experiment.
A group at MIT created a website which presents
you with a number of different self driving
car catastrophes and you can choose which
one you think is the most morally correct.
For example here we have a self driving car that had a sudden brake failure. Should it A, continue ahead to the barrier, killing the passengers, 2 dogs, 1 elderly woman and 2 female athletes
or should it swerve and kill 1 cat, 1 male athlete, 1 elderly man and 1 pregnant woman.
In this we choose between the car killing 2 female executives, 1 man and 2 male executives, or 3 homeless people, 1 woman and 1 man.
The team at MIT collected over 40 million
votes from millions of users around the world,
and some of the results are pretty interesting.
Like most people would rather spare a dog’s
life than a criminal’s, and while Western
countries tended to spare the young over the
elderly, in middle eastern and asian countries
this pattern was much less pronounced.
Anyway, when talking about this, Prof. Procaccia
realized that this data not only revealed
decisions, it could be used to automate decisions.
In other words, this data could be used to
tell self driving cars how to act in a life
threatening situation.
In the words of Prof. Procaccia, they could
create a virtual democracy.
Now what I find so fascinating about this
approach is that it seems so simple when you
hear it but it’s so opposite to anything
we’ve tried in the past.
Most approaches to date have been top down,
in that we try to establish foundational laws
first, like Asimov’s laws of robotics for
example, and then build off those.
This is completely different in that we examine
the opinions of millions and aggregate them
to a final decision.
Now Prof. Procaccia said that this actually
caused a lot of controversy in the ethics
community.
People said things like “we’ve been talking
about this for centuries and you think you’ve
solved it in just one paper?!”
But the whole point of this approach is that
we can deal with The Trolley Problem without
having to solve the Trolley Problem.
I dunno I think that’s really clever.
Now moving onto the more technical stuff (because
I know that’s why you’re here), how did
the team of computer scientists actually implement
this idea and what was the computer science
involved?
I think the main questions are:
How would a self driving car deal with new
scenarios that hadn’t been included in the
Moral Machine experiment?
There will typically be seconds or even microseconds
for a self driving car to assess a situation
and act accordly.
How will it make the right decision so quickly?
How can one algorithm represent over a million
people’s preferences and over 40 million
votes?
Now this stuff is pretty advanced for me so
I actually have a friend with us today who
has greatly studied ethics in AI.
His name is Le and he runs the YouTube channel
Science 4 All which explores AI research and
algorithms, he also has a rather lovely french
accent.
Le, can you tell us how this all works?
Hey Jade!
For sure!
Let’s first give an overview of the process.
It consists of 4 steps.
First the data collection, which as you’ve
mentioned, was done in the moral machine experiment.
Second, the learning step.
The goal here is to use the data from the
moral machine experiment to learn a model
that extrapolates the preferences of each
voter to all possible alternatives.
Third, the summarisation step: combine the
individual models into a single model, which
approximately captures the collective preferences
of all voters over all possible alternatives
in a more efficient manner, to allow for faster
future computations.
And finally aggregation, where we use a summarized
model to run a virtual democracy and come
to a decision.
You already covered step 1 Jade, so I’ll
now start with the next step, learning.
So each voter is given a bunch of scenarios
where they have to choose the best outcome.
Now the options always come in pairs in the
moral machine experiment.
This is because people reason much more clearly
about hypotheticals with 2 alternatives rather
than having to, say, give each situation a
score according to their preference.
For example if someone asked you
“do prefer chocolate or vanilla ice cream?”,
it’s much easier to answer than if they
asked you to score these 100 ice cream flavors
from your favorite to least favorite.
Essentially, you get much more accurate information
from people when you present them with hypothetical
scenarios and ask them to choose.
However, a score is a lot more useful.
You can figure out trends, like
maybe you like sweet ice cream flavors and
don’t like bitter flavors.
Or you prefer to save infants and babies over
adults, or you value human lives over animals.
But luckily machine learning algorithms have
become quite good at inferring scores from
pairwise comparisons, especially if they can
assume that such scores are obtained by combining
the influences of different features, like
sweetness or bitterness.
Or in the case of the trolley problem, such
features may include humans vs pets, swerving
vs staying in line, and so on.
But let’s keep it simple for now.
Let’s just consider 3 factors, age, wealth
and weight.
Say there’s a person who really values the
elderly, the rich and the err, voluptuous.
And assume they value all of these things
equally, so let’s say they assign a weight
of 1 to each feature.
So if we change any of these factors, this
will affect the score..
In this alternative where the person they
could save is skinny, poor and young, they
would give a low score, and when the person
they could save is rich, fat and old, they
would give a high score.
Now, note also that this judgment is what
we have inferred from data.
And thus we should actually be quite uncertain
about our extrapolation.
Plus, many other features that have not been
modeled may affect the person’s preferences.
As a result, we should not only predict a
score, but we should also add that there is
an uncertainty about this score, and we should
even quantify this uncertainty.
Of course, this is probably not how people
would actually score such and such alternative.
In some sense, our algorithm is not trying
to mimic the person’s reasoning.
It rather tries to extrapolate from the data
from the moral machine experiment, how a voter
would act when faced with a new scenario.
Thereby, the algorithm learns a so-called
scoring function.
But the problem is we have too many of these
scoring functions.
In a real life situation, a self driving car
will have seconds or even microseconds to
analyze a situation and act accordingly.
If a million people did the moral machine
experiment, there would be a million scoring
functions, which would take way too long to
sift through and find an answer.
So what the researchers did was combine all
of these functions into one summarized function
which can run in a matter of seconds.
Now of course there is a trade off between
how fast the algorithm runs and how accurate
it will be.
Obviously the ideal situation is to go and
ask each voter what they would prefer, but
because time is of the essence, the best we
can do is crunch all their predicted preferences
into one function which loses the least information
possible.
 
 
Surprisingly the researchers managed to maintain
around 96.2% accuracy.
This means that in 96.2% of cases the summarized
function made the same decision as if you’d
run the full 1 million functions.
Pretty good!
Thanks for answering our questions Le
No worries Jade!
Make sure to check out Le’s channel Science
4 all at the end of this video.
Most of his videos are in French but some
of them have subtitles…
So you might think this is the end of it,
but there’s still one last step.
If we draw an analogy with a presidential
election, we’ve just gathered, or generated,
all the votes, but we still have to count
them.
How we count the votes can make a huge difference
to the final decision, and many countries
have different ways of counting votes in elections.
In the 2016 presidential election, Trump won
even though more people voted for Hillary
because of the way the US tallies the votes.
If the same election were run in France, Hillary
would have won.
This is where the field of Social Choice comes
in.
It’s the mathematical study of political
science and deals with things like choosing
the right voting mechanism to get the most
desired outcome.
In terms of self driving cars, we as a society
can agree on a mechanism with the properties
that we want, for example that it shows no
bias, it’s efficient and it gives people
an incentive to tell the truth rather than
try to manipulate the system.
In their paper Prof. Procaccia shows an example
of a very simple but effective mechanism called
Borda count, which assigns the ranking scores
in an arithmetic progression.
For example, the least favorite option would
be given the score 0, and the next least preferred
the score 1, then 2 and so on.
This is very different to if they were scored
on say an exponential progression, as this
would change the outcome drastically.
So that’s the gist of how it works, but
before we go, Le had some insights that he
was just bursting to share.
I personally find Prof. Procaccia’s work
absolutely fascinating!
I’d argue that it really opens a new line
of research at the intersection of ethics,
social choice and computer science, which
is really becoming crucial for our modern
world.
Indeed, self-driving cars are definitely not
the only automated systems that face moral
dilemmas.
For one thing, autonomous weapons are being
developed.
They have the potential to better target their
victims and greatly mitigate unwanted side
effects.
But they also are extremely scary and worrying.
But there are also more prosaic everyday life
AIs that actually already have huge impacts
on our societies, like recommender systems.
Many studies show that they raise huge moral
issues in terms of privacy, biases or filter
bubbles.
Perhaps generalizations or variants of Prof.
Procaccia’s work could allow to better understand
what ethical values should or should not be
implemented in such systems.
But this raises still another issue.
Indeed, the more complex the moral dilemmas
are, the more it seems that expertise could
be relevant.
Don’t ethicists “know better” than the
layman about the most important aspects of
ethics?
Don’t people eventually change their minds
as they learn more about a particular topic?
In some sense, Prof. Procaccia is laying the
groundwork for a general approach, not about
ethics, but about meta-ethics.
Instead of asking directly what is right and
what is wrong, he proposed a method to determine
what is right and what is wrong.
Perhaps one of the greatest challenges of the
upcoming century could be to determine which
meta-ethics should be used, rather than which
ethics should be implemented.
I don’t know.
But I do find all of this fascinating!
Machine learning is already a huge part of
our lives and will continue to impact us in
the future, so it’s a good idea to become
familiar with the basics so you can make better
informed decision.
If you would like to learn more about machine
learning and algorithms, I would recommend
checking out today’s sponsor, brilliant.org.
Brilliant is an interactive learning website
specializing in math, physics and computer
science.
It covers so many different topics including
an entire course dedicated to machine learning.
This course will provide you with the basic
tools needed to understand the algorithms
and techniques used in everyday machine learning,
including self driving cars.
The same goes for this artificial neural networks
course, which I would recommend starting with
if you’re a beginner looking to learn more.
This course gives you a real sense of how
machines actually learn, starting with the
concepts that link it to the human brain,
leading up to more advanced techniques like
demonstrating how a machine thinks about playing
a game like naughts and crosses.
There are tonnes of other courses which you
can sign up to for free.
However for a true learning experience I would
recommend signing up to their premium membership.
Brilliant is giving a 20% discount to the
first 200 people who sign up with this link.
Just go to brilliant.org/upandatom.
It could make a great Holiday gift.
Thanks for watching guys.
Make sure to check out the video we did over
on Le’s channel about the possibility of
human-level AI by the year 2025.
This is my last video for the year, it’s
been a wonderful year with you guys.
I wish you all a happy holidays and I will
see you in 2019. Bye!
in the new year.
