It's my great pleasure to introduce a phenomenal
panel of speakers that we're going to be hearing
from today on the sort of wide topic area of
behavioural insights from around the world.
So just to quickly introduce myself, I'm Kate Glazebrook.
I'm the CEO and one of the co-founders of Applied,
which is a tool spun out of the Behavioural Insights Team
to use behavioural science to improve hiring decisions.
But this is not about hiring decisions,
this is about the way in which behavioural science is
being used increasingly in more and disparate
parts of the world.
So we're going to hear from three fantastic speakers,
and the kind of run through of the session will be as follows.
First of all we're going to be hearing from Filippo Cavassini.
We're then going to be hearing from Varun Gauri.
And then we're going to hear from Lisa Cameron.
And they're each going to do a presentation of about
15 minutes.
They're going to aim to focus on the way in which
they're using behavioural science to drive new
insights about how behaviours are influenced
by different contexts, and how these kinds of
burgeoning fields of behavioural science,
cognitive, neuroscience and social psychology,
are playing out in different parts of the world.
Many of you have probably been in sessions already
today and yesterday about WEIRD populations and the
way in which most of our research to date has been
formed on a series of populations,
usually North American or across the Atlantic.
And fantastically we're going to be hearing today
from people who are breaking boundaries in that space.
So without further ado, I'm going to introduce our
first speaker, Filippo.
So Filippo is the economic advisor of the
Public Governance Directorate of the OECD.
Filippo has 15 plus years of experience working in
government and international organisations.
At the OECD Filippo has been co-leading the work of
the organisation in the area of developing the
application of behavioural insights to public policy
and has coordinated and co-authored the now very
important study, behavioural insights in
public policy, lessons from around the world.
Who better to start a session like this?
It's a global mapping of applications of behavioural
insights to public policy in different contexts.
Filippo has also co-led the development of the OECD's
work in the organisational performance independence of
regulatory agencies.
He also works extensively on good regulatory practice
and public sector reform in Europe and Asia.
Prior to joining the OECD Filippo worked for a number
of years at the World Bank, advising governments in
Central Asia, Europe and Latin America, and in the
French National Assembly where he was also a member of
the team advising the President of the National Assembly.
He holds a master in public policy from
Harvard Kennedy School, and we look forward to hearing
his presentation.
(Applause)
Well thank you very much Kate.
It was an amazing introduction,
I think probably one of the best I ever got.
I'm particularly glad to be here.
It's both my first time here in Australia,
so particularly excited to be with you today at BX
presenting some of the work we have been doing at the OECD.
How many of you, do you know what the OECD is?
Could you raise your hands?
Oh wow. So you already know everything, so I won't give
you the speech on what the organisation is.
Essentially it's an ... constitution bringing
together 35 member countries.
We don't implement policies, we don't design
policies ourselves.
We work with our member countries to collect
experiences in designing policies, and finding
solutions together to common programs.
One of the areas we have been working on is
improving decision making processes,
and in particular in the area where I work on, is really
about regulatory policies, how to improve regulation.
This is where we started, if you like, in 2012,
together with another colleague who couldn't be
here today, Faza Naru, because actually his work
on behavioural insights, the one we did together,
earned him a promotion.
So now he's the Chief of Staff of the Director for
Corporate Service at the OECD, applying behavioural
insights to the organisations.
So stay tuned.
Probably he will have also insights on how the OECD is
applying behavioural insights to itself.
But we started in 2012 looking at the way in which
behavioural insights can improve regulation,
by doing the first mapping of some of the applications,
and then as Kate was mentioning, we did a global
mapping of application of behavioural insights across
the world, which was published in 2017,
which was a collection of over 100 case studies of
applications of behavioural insights in many different
areas where governments work and intervene.
What I'm going to show you today is part of -
it's almost a spin-off of that report.
The report is out there, has been widely downloaded.
I just wanted to show a bit of a follow up we did based
on the report, focused on the institutions which are
applying behavioural insights across the world.
And then I'll tell you a little bit of the kind of
work we are doing, show you probably a bit of the next
frontier in terms of work which is being done in
other OECD countries in applying behavioural
insights to public policy, and then some of our thinking,
and the thinking we are developing together
with OECD members and partner countries,
on what are the next steps and how policymakers can be
supported in applying behavioural insights to
public policy.
So what's going on? Quick experiment.
How many behavioural insight units exist in the world?
Guess.
Forty?
Right, we have 40.
Eighty?
Eighty.
One hundred and fifty?
What was - 200? A hundred and?
Fifty?
Fifty? Right. Anyone else?
Ten?
Ten? Okay. This was of course a highly sophisticated
behaviourally informed experiment.
I think we got many different answers.
Someone got it right.
Interesting, because maybe we could try to actually
make it an experiment and see whether those who
replied were actually coming from behavioural
insights units, those who replied in a certain way,
and those who replied in a different way were coming
from other parts of government.
But I'll give you the answer, and I give you the
answer through a map which actually put, on the map,
201 institutions which are applying behavioural
insights across the world.
These actually include - the last institution we had
was actually on Friday before I boarded a plane to
come here, which is an institution in Singapore,
within government.
What we did there was actually to map
institutions within government, nudge units,
so units which are really specialised in applying
behavioural insights.
Think about BETA, think about behavioural insights
unit within line departments here in Australia.
But those are units which have other functions,
economic analysis, and also include behavioural insights,
applying behavioural insights when
developing analysis and public policy.
We also wanted to include institutions outside government
which however work with government in developing
policies which are informed by behavioural insights,
and we have a number also there.
Plus international organisation, multilateral organisation,
like for example, the World Bank.
And we included also the OECD which has been doing
a little bit of work on behavioural insights.
So this shows, probably in a snapshot I would say,
that behavioural insights is definitely taking root
across the world.
Just to give you a bit more details on where these
behavioural insight units are located.
As you can see, they are really literally across government.
Here we tried to map a little bit the areas,
the policy areas, which are applying behavioural
insights based on the institution, and we track
roughly 25 policy areas.
Where are these institutions located?
Well, as you can imagine, I think quarter of them
within government - not surprisingly, since we have
application to government - but also of course some
almost 20 percent outside government, and a few in
multilateral organisations.
And what is the institutional setup?
This was, if you like, a hypothesis we had developed
when doing the case study collection which was
published in 2017.
What we saw were actually different institutional setups.
So you can have sort of almost a central steering unit.
Think about BETA here in Australia, think about BIT
which originally was located within the Cabinet office
in the UK, which would be the Prime Minister
and Cabinet office here in Australia.
Now it's been put outside government, but still owned
by and powered by the Cabinet office.
This will be the sort of central steering we had in mind.
You could have also diffused units, and again
this is the case, for example, in Australia where
you have line departments who've actually started
applying behavioural insights.
And you can add project base, simply teams which
come together, and these can come together through
different department, but across different departments,
to implement behavioural insight to a specific policy,
to address a specific policy problem.
And I think the interesting thing is that the three
models can actually live together in the same
institutional setup.
Just to give you a bit of the distribution across the
201 institutions we mapped,
a number of them are
actually behavioural insight unit either mostly
within line department - there are a few nudge units
at the Centre - and then some project based activities.
Alright, now you can say well, very nice, here is
the OECD which gives us, you know, the wide snapshot
of what's going on around the world.
And so what?
So given that behavioural insights has taken root
across the world, what is also the next steps,
if you like, in applying behavioural insights?
I can show you a little bit some of the work we are
doing with some of our OECD members.
First, probably something in a situation where
behavioural insights could have been applied.
This is from Ireland, 2014.
Ireland introduced water charges.
So until 2014 there were no charges for water in Ireland.
Ireland is part of the European Union.
The European Union, there is a directive which says
that member states have to recover the cost of
delivering water.
And so Ireland was actually going through the risk of
an infringement procedure, which means that it would
have been punished by the European Union if they
hadn't applied this directive, and so they did.
Essentially they introduced water charges, and there
were riots in the street.
Why? Well probably because in part the reform and the way
in which the reform was being carried out,
had probably not been explained sufficiently to citizens,
but also probably because it had not been framed.
Had they been thinking about also the way in which
framing prices for citizens - and these were citizens
who had never paid for water before -
probably the consequences would have been different.
Well let's move to another country, if you like
another Celtic one, Scotland.
Here we are working actually with a regulatory agency,
the Water Industry Commission for Scotland,
to actually use behavioural insights to gather evidence
and views and true preferences of consumers,
when regulating water.
So in Scotland there are water charges.
Every six years the regulator works with the
regulated entity - there's only one company, state owned,
Scottish Water - and the government to set up
water charges.
And you can think of a regulatory process
a bit as a triangle.
Government, regulator, regulated company.
At the centre you have the citizens.
I mean citizen can be a very elusive term.
How you gather the evidence and the views of the citizens?
In Scotland they set up what is called a customer forum,
the mission of which is really to elicit the
views of consumers.
Not to represent them - there is a statutory body
which represents and advocates for the rights of consumer -
but rather elicit the views.
And what the customer forum is doing, and we are
working with them together also with the price lab led
by Dr Pete ... in Ireland, to really experiment around,
for example, framing prices and the understanding of
consumers of prices, but those who engage in the
true preferences of consumers when taking
decision which then feed into the price setting process.
Another area where behavioural insights could
have been applied is around safety.
So this is 2010.
This is the Deepwater Horizon oil spill when an
oil rig exploded in the Gulf of Mexico.
Well there was clear evidence that that company
had not been following safety requirements.
In some of the reports which were issued after
the accident, there were actually clear references
to the fact that there was a limited or poor safety culture,
and that had in part contributed to that accident.
This is a serious issue of course, and together with
some of the regulators we work with, we have been
thinking on how to develop an organisational culture
which actually supports safety and compliance with
safety requirements.
In other words, we wouldn't be just nudging individuals,
but the idea is see how we can nudge an
entire organisation, a regulated company,
to respect safety requirements.
We are actually right now in the middle of the process.
So we did first a scoping of the possible barriers to
safety culture and the enablers to safety culture
in terms of behaviour.
We are now doing testing around three areas.
One is awareness, how you create awareness of safety
within the organisations, share responsibility,
how you create share responsibility within the
regulated entity, but also between the regulator and
the regulated entities, and how you change the
attitudes of leadership towards actually respecting
safety requirements.
And they are the key questions we are asking ourselves,
is what behavioural change at which point
of the decision making process can really
make a difference?
You can think of an organisation as a
collection of individuals, true, but organisations are
also part of the context, they have a mission,
and they also have decision making processes.
How you can change the choice architecture within
an organisation so that it can really affect the
culture of that organisation?
We will be running experiments in four different
countries and working with four different regulators -
Mexico, Oman, Ireland and Canada -
and we expect to have results towards the end of the year.
So this is a bit the next frontier.
So what does it mean for governments?
And I'm going to conclude.
I've been nudged towards the end of my presentation.
Well I think one of the key insights - this was one of
the key insights from the 2017 report.
And yesterday I was here in this room when Kate Phillips
from the Victoria government was presenting
actually the next steps, and highlighting the
importance of actually thinking about behavioural insights,
not just at the implementation phase,
where mostly behavioural insights have been applied,
but also the design phase.
This is probably part of the new frontier, how you
make behavioural insights a tool which is used from the
very beginning when designing policy,
already to scope the problem.
It is an area which is extremely important, and probably
extremely also promising for behavioural insights,
and ultimately how to embed it into the
policymaking process.
I think this is a bit our research question,
and we have been working on it by developing guidance on
applying behavioural insights and working
together with practitioners to develop a guidance and
sort of a policy tool kit, an ethical framework,
which could help governments apply mainstream
behavioural insights.
The idea is to develop a tool kit, an ethical framework
which is for practitioners by practitioners.
And we will be discussing a draft in a conference we
will be organising jointly with the Western Cape
Government at the end of September in Cape Town,
and of course you are all invited.
I'll give you a sneak preview of the framework.
We have been working with Dr Pelle Hansen from
Roskilde University in Denmark, also one of the
founders of the European nudge network and I Nudge You,
which is an outfit working with public
institutions in Denmark in applying behavioural
insights to policy.
I mean this again, it's an acronym, we call it BASICS -
behaviour, analysis, strategy, intervention, continuation.
Alright, you might say this is yet another tool kit,
and we haven't finished yet so we will be
keen to have also your feedback while we're developing.
But I think one of the thinking we are developing
is really how to think of the critical decision
points where behavioural insights can be applied in
the decision making process.
When you develop policies, you first scope the problem.
What can be the role of behavioural insights in
scoping the problem?
And then you start to line up solutions, and again,
what is the role of behavioural insights in
deciding what solutions can have the greater impact?
But then you also want to think about implementation
and how thinking about implementation already in
the design phase can actually help you improve
the effectiveness of policymaking.
So this is a bit the thinking we are developing.
I'll leave you with this, and also of course with
a save the date for the conference to allow me also
a piece of advertisement.
And so thank you very much indeed and I'm looking
forward to the debate.
(Applause)
Thank you Filippo.
Fantastic beginning to the conversation.
So now I have the great pleasure of introducing
Varun Gauri, who is the senior economist in the
Development Economics Vice Presidency at the World Bank.
He co-leads the Mind, Behaviour and Development Unit
which integrates behavioural science into
the design of anti-poverty policies worldwide.
He was the co-director of the World Development
Report 2015, Mind, Society and Behaviour, which I
understand have been one of the most highly
downloaded World Bank reports ever.
Maybe you used some nudge techniques on people there.
And he serves on the editorial boards of the journal
Behavioural Public Policy in Health and Human Rights,
and is a member of the World Economic Forum Council
on behaviour.
His research spans widely, so I'm going to let him
talk to you a bit about what he's doing at the
World Bank, but his current research is investigating
the influence of social norms on women's economic
decision making, compliance with judicial
human rights orders, and the influence of corporation
and identity on ideas of distributive justice.
So just a few, you know, cheap and easy things
to be working on.
We look forward to hearing from him,
and I'll hand over to you Varun.
(Applause)
Thanks so much Kate for that kind introduction.
It's great to be speaking to all of you.
So as Kate mentioned, what we try and do in the
World Bank is argue that behavioural insights is
really useful for development policy.
I also want to make the general claim, which I'll
try and bear out in the talk, that development policy
is really useful for the behavioural insights community.
Because when you start working in the non-WEIRD
countries and countries with very high levels of poverty,
the kinds of questions you ask, the kinds of
measurements you need, the kind of programming
you implement really begins to change.
And I think it sort of illuminates some of the
questions that the community is trying to address.
The Mind, Behaviour and Development Unit,
or MBeD as we like to call ourselves, is a team within
the World Bank whose mission it is to use
behavioural insights to improve development effectiveness,
to promote the use of behavioural insights
within governments directly to build capacity,
and finally to provide evidence about what works.
As I mentioned, we also try to innovate and develop new
measures that are appropriate to the tasks at hand.
I stole this slide from Filippo, thank you very much.
Behavioural insights is now all over the world,
changing day by day.
I know we can add a couple of institutions in South Asia
that aren't currently covered, so maybe we're
at least at 203 at the moment.
Our team is about two years old.
We followed the World Development Report 2015.
We have ten people full time.
We have 26 projects for which we have evidence.
And you're welcome to go to our website.
There is about ten results briefs which summarise
our findings.
I'll talk about some of those.
Some of them are in paper, some are in briefs.
We work with a lot of different partners,
78 by last tally, in implementing our work.
We have 87 total projects.
That means we have about 60 that are currently ongoing
in 65 plus countries. So we're pretty busy.
What I wanted to do is just share with you about nine
or ten projects that we have some results on or are
currently working on that we're pretty excited about.
The list of partners.
So the first one I'll talk about is improving student
performance in Peru.
Here we looked at the question of what can help
seventh grade students in Peru, in Lima, you know,
in poor areas around the country, and we're
interested in the use of the growth mindset.
Some of you may know Carol Dweck's work and that work
of her collaborators.
The basic idea is that young kids, or any of us really,
but especially in this context young kids,
particularly if you're poor, develop the idea that
your intelligence is fixed and so why bother to study,
why bother to work hard, you're not going to go anywhere.
It's kind of a fatalistic attitude.
This is what's sometimes called a fixed mindset
as opposed to a growth mindset.
And you can actually, in her work, teach the growth
mindset and help students understand that if they
actually exert effort, intelligence may change,
they may grow.
The key idea here, I think that when we typically
think about students and learning, we think,
you know, either you're smart or you're not,
or you try hard or you don't. Right?
So if you're not smart, there's no hope for you.
If you don't try hard, you know, some parents will
spank their kids, or give them an iPhone if they do well.
You know, it's a question of incentives or natural talent.
And the key idea of the growth mindset,
no, it's actually a mental model of learning, so that when
you're studying for a test you need to sort of think
about your strengths and weaknesses,
and think my weaknesses can actually be addressed.
Since it's World Cup season, you know, it's sort
of like being a footballer and you realise I've got a
strength with my right foot but my left foot is weak,
you just don't think I'm just stuck, I'm never going
to learn to use my left foot.
You practice.
And that's the growth mindset.
You actually put it into play.
So what we did in Peru, initially with 50,000
students then with 250,000 students, was to have them
learn a little about neuroplasticity, which is
how growth mindset is often taught, the ideas that
neurons rewire in the brain throughout life.
The kids saw these little posters in Spanish.
They also wrote little essays about what they're
going to do in this process.
Some of the posters around the classroom were the idea
that the brain is like a muscle, you know,
you exercise it and it gets stronger.
And then this was a two hour lesson.
The materials were sent out from the Ministry of Education
to a number of schools, at first 50,000 students,
then second year 250,000 students.
About 60 to 70 percent of the schools were able to
implement the lesson.
About a third of the time the materials were lost in
the mail or things didn't really happen, you know,
so we weren't really sure what we were going to find.
Two months after this was implemented, the kids took
a standardised test.
So outcome measure was performance on the
standardised test.
It was, you know, between treatment and control.
We used randomisation. It was an RCT.
So the findings were that even if you look at schools who,
you know, everyone that received instructions
but didn't implement it, within two months -
this is for kids outside Lima - they scored about
0.1 to 0.12 higher standard deviations in their
math and reading test scores.
So that's a lot in two months, right?
That's about four months of learning.
The lesson cost about 20 cents per student to implement,
so you got about four months of learning for
20 cents per child.
It was a pretty impressive finding we thought.
So now we're taking it actually to around the world,
the Western Cape Government, you know,
who we're in touch with as Filippo mentioned.
They're implementing it, and in Indonesia we're
doing a trial of about
250,000 students as well,
where the students there are getting similar lessons.
So that's one result that we think is quite exciting.
Another is tax compliance, and this follows on, you know,
the pioneering work of the behavioural insights
team and others, trying to increase tax payment in Poland.
Some of you probably know the work of - you know,
the BIT work early on with 150,000 late tax payers.
With BIT we subsequently did it in Guatemala,
and found again that a little nudge and social norms
letters makes a difference in that study.
Although I will add at this point we've now followed
those people who were nudged with social norms
letters in Guatemala a year later, and there actually
is an impact one year later for people who were only
nudged once in the year prior.
So it affected their tax payments in that year,
but then also a year later.
And this is a little bit different.
I remember Cass Sunstein saying yesterday, you know,
quoting some of the Todd Rogers, Hunt Allcott work,
saying that social norms, you need to put it in for
two years to make a difference in the energy
conservation context.
There's some evidence that it might be a little bit different,
and I don't know exactly why, but the thought is that
when you lower your temperature of your house,
that's sort of not a meaningful experience.
But I think actually paying taxes itself is an
experience that you've been nudged to do, which then
may have an effect a year later.
That's the thought.
Anyway, getting back to Poland, we designed nine
behaviourally informed letters, changed the first paragraph,
implemented it, and this is the main finding.
Overall the hard tone letters resulted in a
15 percent increase in payment rates compared to about
7.5 percent in the soft tone.
In the UK paper that you may remember, and in the
Guatemala paper too, the social norms letter,
which is the sort of soft tone letter, had the highest impact.
That's not what we found in Poland.
We're not exactly sure why.
The main takeaway, I think as David Halpern said,
is test, test, test. We don't know.
It could be for instance that in Poland people had
been hearing about - it's a post communist country.
They've been hearing about social norms for a long time.
It's just not jumping out at people as much in the same way.
We're doing some other work in Latvia and possibly
Moldova on tax compliance.
This is some work in Tanzania, trying to use
a mobile platform to encourage savings.
Not a large study.
The basic takeaway was that social influence was able
to increase savings, but some of the other things
like a simplified SMS and focusing on agency
as a message actually lowered savings rates.
Mental accounting didn't really have that much of an impact.
I'm not quite sure what to make of this.
This is something we turned into a blog rather than
a results brief, because we weren't quite sure what to
make of the findings.
But it's out there, it's public.
Motivating health care workers in Nigeria.
So the World Bank for some time has been working with
countries throughout the world on public
expenditure tracking.
The concern here is that resources, when allocated
from the Centre, don't make it all the way down to the
clinics or to the schools.
And if you can actually track what's supposed to go
there it actually improves outcomes in the end.
There's a famous paper in Uganda in the education
sector that showed that actually just posting
what's supposed to be at the school helped improve
outcomes in schools in education.
But the problem is that even filling out
expenditure tracking is a problem, right?
So how do you motivate people to actually do the
tracking which you think is going to make a difference?
So we tried a nudge approach with 140 clinics
in Nigeria in two states, in Ekiti and in Naija.
In one treatment, every time someone appropriately
filled out a tracking form they got a lottery ticket.
In another one they got a star, and if they got
enough stars they eventually got social
recognition and had a chance to sort of,
you know, meet an important public official.
So it was the social recognition and a lottery.
The lottery, you would win a smartphone eventually.
The winner of the lottery would win a smartphone.
So the lottery did not have an impact in our study.
The social recognition did, at least in Ekiti.
It increased the accuracy and quality of the tracking
by 13 percent.
And this is a paper we finished.
It's one of our results briefs.
Generally speaking, you know, motivating health
care workers is important, using social recognition
is important, and we're lining up some more work in that space.
So improving sanitation.
And Lisa's going to speak about this.
Some of her work has been really important for the
design of what we're thinking about.
But I'll situate this in the context of measuring
social norms.
We're really quite interested in measuring social norms.
One of the differences in the kind of work we do,
I would say, is that the diagnostics are so crucial,
because when you're going in to a very different context,
a non-WEIRD context, you've got to spend
a lot of time understanding what's going on.
And social norms, you know, we all know they're crucial
for gender issues, for corruption and tax payments
and all kinds of things.
So within that space we have a project trying to
measure social norms relating to gender force participation,
labour force participation on the part of women
in Jordan, which really sort of tries to
carefully measure it. That's a larger study.
This is a smaller study in ..., India, where in five villages,
in about 300 households - it's a small study -
we tried to first measure the prevalence of
social norms against open defecation.
And the basic finding was that - among toilet owners,
because the government of India has a large program
to increase latrine ownership, but in the past
what often happens is that people will buy latrines
but not use them right, and they end up being used for
grain storage or something else, either because it's
a collective problem, or it's a social norm, or people
don't necessarily experience the health
benefits of using them unless lots of people in
the community also do so.
So when we went out and measured the social norms -
and the measurement was what do you think other people do?
What do you think other people think you should do?
Those two parts, empirical and normative expectations -
the finding was that people guessed that latrine
owners in these villages were using latrines about
half the time.
In fact it was 80 percent of the time.
So they were underestimating latrine use
on the part of toilet owners.
Why? Perhaps because it's visible when people will
defecate in the fields, but not so visible when people go,
you know, in a latrine.
When they were asked about the normative question they
said yes, you should definitely be using a latrine.
But then we asked well, is it okay to defecate in the
fields if you're out, you know, in the middle of the day?
Oh yeah, well that's fine.
Is it okay if you need to sort of like, you know,
be with someone else? That's fine too.
So they covered all these exceptions even though
there was a norm against it.
And so in the intervention we designed three posters.
The first one was, you know, on the left,
just showing that 80 percent in fact use a toilet.
The middle one was sort of like, you know, talking about -
the middle two and the right one were talking about
there's this concept of, you know, what do
people associate toilet use with, and in fact it was
perceived to be dirty, not clean.
Even though it's a sanitation problem,
but people think of it as sort of dirty, because,
you know, they are dirty.
So we tried to change the framing about it, you know,
it's part of good hygienic practice, in the right
two posters.
So these were randomly applied to different villages,
some with just a mass media campaign and
some with a norm entrepreneur identifying
someone who's going to actually multiply the message.
And the finding was that both the norm entrepreneur
treatment and the mass communication treatment
resulted in lower open defecation rates,
self-reported open defecation rates,
just after a short intervention.
Absenteeism in Peru of teachers.
This was joint with the behavioural insights team
as a partner.
Messages. Two versions of emails were sent to both principals
and teachers to try and reduce absenteeism.
There are results.
The finding was that it had an impact for school directors,
about three or four percent, but no impact for teachers.
You know, and we're not exactly sure why.
You know, the messaging might not have reached the teachers,
the measurement might not be right.
We're not quite sure. But some results there.
So now here's a couple of projects that are ongoing
that we're pretty excited about.
We did some drop outs in Guatemala.
From the primary to secondary transition rate
in Guatemala, about 30 percent of kids drop out,
and then only 50 percent will eventually graduate
altogether from high school.
But that period, that year, is huge.
So we're working with the Ministry of Education and
we're targeting every school in the country.
Another World Bank team has sort of developed a model
to predict who's likely to drop out.
So the idea is that there'll be a control group
of school directors, others will get this list of
students who have been identified to be at risk,
and then a third group will get reminders as well as a
social recognition treatment to see if we can
actually get them to talk to parents, talk to kids,
to reduce drop out rates.
You know, that's in the field.
We're starting some work on energy efficiency,
energy efficiency behaviour in South Africa with the
Western Cape Government.
We've done some work in the ... electricity bill payment.
And then we're starting some work in a number of
African countries on the uptake of low carbon technologies,
off grid solar and biomass indoor cook stoves.
And these are private firms that are marketing these
products that people don't trust the warranty,
they're new technologies, so how do you increase take up
of new technologies?
That's sort of like the larger question that we're beginning.
Cash transfer modernisation in Bangladesh.
So the government there is - there's a huge sort of
cash transfer program for widows, for disabled,
you know, and it's in cash but they're sort of moving it
all digital.
So we're going to think about an opportunity to
sort of like nudge with the digital work, but also in
particular we're going to work with the unions or the
low levels of government social workers who are
sort of just kind of overwhelmed.
You know, sort of like there's so many different
things going on, so how can we sort of add bandwidth to
help them make better decisions about what they do?
And we're also starting some work on supporting
community health workers.
This will be the motivation that I mentioned.
But also again adding cognitive bandwidth.
And in Cameroon one of the key challenges is,
among the many, family planning.
You know, because community health workers are
often illiterate, they're dealing with young girls
who don't quite know about contraception and the
options available, and how is that person going to
provide appropriate advice and advice that's tailored
to the girl who's coming in?
You've got to know the clinical conditions of the girl.
You've got to know whether she's, you know, married or
likely to be married, her own long term preferences
about how many kids to have, when to have them,
and then you can steer that girl towards, you know,
a short acting contraceptive, a longer acting
contraceptive like, you know, an injectable or an IUD.
So we've developed a tablet based app that's going to
guide the community health workers to help them do
these diagnostics more accurately to help provide -
because when you present the options to young girls,
what you say first matters, so it's going to be like
leading them toward the kind of thing that might be
best suited to their particular clinical
condition and their personal needs.
So we're doing all kinds of work in lots of
different countries.
It's very exciting and it brings new kinds of
questions to the behavioural insights community,
and we'd love to partner with you in all the work.
Thanks so much.
(Applause)
Thanks Varun. Well you talk about cognitive bandwidth.
How do you keep on top of so many amazing projects
all at once?
So third and finally I'm really thrilled to
introduce Lisa Cameron.
She is a professor at the Melbourne Institute of
Applied Economic and Social Research at the
University of Melbourne.
She is an empirical microeconomist and
experimental economist with two decades of experience
conducting economic policy experiments with a focus on
poverty alleviation and policies that impact
disadvantaged groups.
She holds a PhD in economics from Princeton
and she's an affiliated professor of the MIT's
J-PAL group, which is a network of researchers
conducting randomised control trials and
evaluations of policy interventions around the globe.
And after Lisa speaks everyone will be joining me
on the stage for a conversation, so please
keep your questions in mind and we'll certainly have
time to delve into those.
But with no further ado, Lisa.
Talking about behavioural economics and randomised
control trials with some examples from Indonesia,
from research I've been involved in.
So as Kate said, I conduct evaluations of economic
and social policy.
That's part of my research agenda, a big part of it.
And I'm interested in marginalised and
disadvantaged groups.
And I've been working to date predominantly in
South East Asia, mainly Indonesia, China and other
South East Asian countries.
And in the last year I moved to the Melbourne Institute
which has more of a domestic focus.
So I'm now also developing projects in the
Australian context.
So I'm going to be talking about two randomised control trials.
I also do lab style experiments trying to
elicit behavioural preferences, but I'm not
going to talk about that today.
I'm also an affiliate of J-PAL, which is MIT's
Poverty Action Lab, which is a network of
approximately 150 academics at 52 universities around
the world who conduct randomised evaluations.
And here we have a map, and you can see J-PAL has its
main office at MIT, but it also has a number of
offices around the world which facilitate the
running of randomised control trials.
All of my work with J-PAL has been conducted through
the South East Asia office in Jakarta, which is funded
by the Department of Foreign Affairs and Trade
of the Australian government.
J-PAL researchers work across a whole range of
different research areas.
These are their main research areas, so you can
see agriculture, crime, violence, education,
environment, finance, gender, health, labour markets,
political economy in government.
So it's a very wide ranging research agenda.
So the two examples I'm going to talk about today
are first, community led total sanitation, which is
a very popular approach to improving sanitation in the
developing world, and I'll talk about the details of
that approach in a second.
And so that was a randomised control trial.
You can think of CLTS as being a behavioural nudge
in a way.
It's a bit more of a shove than a nudge.
But you'll see when I talk about the details.
And I particularly like this randomised control trial,
because it speaks directly to policy design,
as you'll see when I present the results.
It's also a global evaluation.
One of the criticisms of randomised control trials
is that external validity can be an issue.
So that means when you run a trial in one location,
are those results valid in another location
or a different context?
And so this particular evaluation was part
of a global evaluation.
So I led the Indonesian evaluation, and there were
simultaneously evaluations being conducted in India.
There was one in Mali which wasn't completely
coordinated with ours, Tanzania, and then there
were some handwashing evaluations also being
conducted at the same time.
So we were looking at ways to improve water,
sanitation and hygiene in a number of different
locations around the world at the same time using
standardised survey questions so we had the
same kind of information in each of those locations.
And also this was a randomised control trial of
a program that was being run at scale.
So often when you're evaluating policies or
behavioural nudges, often it's being piloted and
you're evaluating a project that's being implemented
very carefully, and so it's questionable as to whether
the results from those kind of evaluations will be
applicable when a program is scaled up.
So this particular evaluation was conducted at
scale across rural East Java.
And then the second example, if I have time,
is going to be looking at an information intervention
with Indonesian women who expressed an interest in
working abroad, which is normally in the Middle East
or elsewhere in Asia, where they're often treated
pretty abysmally.
And so we have an intervention which we evaluate,
but we also actually designed and implemented ourselves,
the research team, which provides information to the
women so that they can make better informed choices.
That's in progress, so we don't have results for that one yet.
So the first example, sanitation.
So as I said, it's an evaluation of an at scale intervention.
This was with the World Bank.
So there's a global evaluation with funding from
the Gates Foundation, and also for the Indonesian trial,
my co-author, Manisha Shah and I got some additional
funding from the Australian Research Council.
I'm going to focus mainly on Indonesia, but there
were similar evaluations in Indonesia, India,
Tanzania and Mali.
And there was an extensive data collection effort.
So we weren't relying on administrative data.
We were actually collecting our own survey data,
which involved running surveys which went for about
90 minutes in about 1,200 households across
160 communities in Indonesia, in East Java.
And we were focusing on child health impacts.
So the aim of community led total sanitation is to
build toilets, but the ultimate aim of building
toilets is that people use that, so you'll have less
faecal contamination and you'll end up with better
health outcomes, particularly for kids who
are very susceptible to getting diarrhoeal disease.
And so we surveyed 1,200 households that all had
children under the age of two, and we followed the
households over a two year period to see if we can
discern any improvements in child health associated
with this program.
So CLTS is an interesting program.
So it aims to stimulate demand for sanitation.
So it's not about providing toilets to households
or to villages, which has been tried and been shown
to be largely ineffective.
So what CLTS does is it basically involves facilitators,
so one, two or three people go out to a village
and they hold these really graphic, shame inducing
community meetings in which the community
analyses the existing sanitation practices and
the negative health consequences, but in a very
kind of graphic way.
And so it's basically designed to shame people
into building toilets.
And so you can see, this is some of the advertising
material that goes with CLTS in Indonesia, and you
can see these two people kind of gossiping about
this fellow here who defecates in the open and
is surrounded by flies and so forth.
And so what they do is it starts with a mapping of
the village, where you have on the ground the kind of
rough map of the village is drawn, and then community
members are asked to show where they live and then
where they defecate.
And they kind of trail sand from where they live to
where they defecate and back, and at the end of
that whole process you've just got this mess of sand
on the map, basically showing that faecal
material is just being tracked through the village
as people go to defecate in the open and come back.
They conduct a walk of shame during which people
indicate where they defecate and where they
obtain water from, and so the idea is that, you know,
people are shown to be defecating - in Indonesia
it's in the rivers mainly.
So in the river here - and it's pretty rare to get
drinking water from rivers in Indonesia, but you'll
have kids swimming nearby or people washing their
clothes nearby, so it just highlights the, you know,
transfer of these germs.
And so after you have this kind of graphic discussion,
they calculate how many kilograms of faecal
material are produced in the village each week.
And so, you know, basically it exposes that everybody's
eating everybody else's poo.
Fundamentally that's what it does.
So I went to one of these.
I've seen videos of these events, but I also went to
one or two of these events, and you see people giggling
and kind of being embarrassed, and the idea
is that that motivates the community to forge their
own plan to improve sanitation.
But they're kind of largely left on their own at that point.
So there's no money.
In Indonesia, in some instances you had one follow up visit,
but the community's basically left on its own.
So it's a very low cost approach to improving sanitation,
and so it's been used in over 60 countries
around the world, and thought to be very successful.
But it hadn't been rigorously evaluated
at the time that we conducted this RCT.
So we randomly selected 80 communities as our
treatment communities and 80 communities as our
control communities.
I always think when I put up this figure that it
looks very neat, but if anyone's run a randomised
control trial you know there's a whole back story
to these kind of diagrams.
And this involved road trips, negotiating with
ten local governments as to whether they'd be willing
to randomise, and then of course they had ideas about
which communities they wanted to go to
with these programs, as you would expect.
And so we negotiated that they would give us a list.
If they were going to go to 20 communities,
we negotiated that they would give us a list of at least
30 communities they'd be happy to go to, and then
that allowed us to choose ten communities from each
district as our controls, and then we also chose ten
as the treatment randomly.
So we ended up with control and treatment communities
scattered across East Java, all in rural areas.
And so we run our baseline survey, and then the
program goes into the treatment villages,
and then we come back two years later and we interview the
same households in the same villages,
and this is what we found.
We did find improvements in toilet construction.
So we had 14.9 percent of households in treatment
communities building toilets, which sounds like
quite a lot over a two year period.
I think that is quite a lot.
And the implementers were very much of the view that
this was a great program and they saw all of this
toilet construction, but they weren't going to the
communities where the program wasn't being implemented.
And so we went to the control communities and ran
our surveys, and in those control communities
12.5 percent of households were building toilets.
Because you're in an environment where you've
got economic growth, you've got all sorts of economic
development occurring, and so people are building
toilets of their own accord.
So in fact this very kind of popular program,
it's not to say it's not effective, it's just that
it wasn't terribly effective.
It increased toilet construction.
It's 19 percent, but it's only, what's that,
a 2.4 percentage point increase in toilet construction.
And then what we did is we looked at who was building
the toilets in the treatment communities,
and we saw that actually the poorest were really struggling.
So there was no difference in toilet construction of
the poorest in the treatment communities and
those in the control communities.
So this suggested to us some feedback to the
policy framework, where - CLTS has this strong view
that you should not subsidise toilets, but our results
are showing us that well, maybe you want to think
about subsidising them, or at least providing some
other kind of support for the poor so that the poor
can build toilets.
We looked at health impacts, as I said.
We collected faecal samples from
1,200 children approximately,
and we did find a decrease in worm infestations
amongst the children in the treatment groups,
and it was quite a large decrease, 44 percent
decrease in infestations.
And then we looked at child height, we also looked
at anaemia, and we looked at cognitive development,
social development, and we found nothing further down
that behavioural chain.
So that was kind of interesting, if not a little disappointing.
And so what's also nice about this evaluation is
that it led to a randomised control trial of CLTS in Laos,
which is now trying to explore what we found in
the Indonesian context.
So we found that poorer households couldn't
build toilets, or were struggling to build toilets.
So in Laos we're running an RCT of the same program
where we've got different treatment arms where we're
trying to incentivise communities to either
support the poorer households in their
communities to build toilets through providing
payments to communities where everybody ends up
with a toilet.
We're also providing reimbursement in another
treatment arm to poor households for the cost of a toilet,
and we're also incentivising the community facilitators -
so they're the people that are kind of involved more
closely with the program within the community -
so that they get a small payment for each
poor household in the village that builds a toilet.
So that's an RCT that's in progress that will provide
us with more understanding of how to overcome the
difficulties that we uncovered in the original RCT.
As I said, it was part of a global evaluation.
And so here's Indonesia.
And we saw very different rates of toilet
construction in the different countries.
So Indonesia was actually the smallest increase in
toilet construction, then we have India where they
had CLTS, but they did have subsidies.
And in India you did have a higher rate of
toilet construction.
Tanzania was very similar to Indonesia and has a kind
of modest rate of toilet construction.
In Mali they had CLTS and they had monthly visits.
So they had a lot of follow up, so 12 follow up visits,
and that resulted in a very large increase
in toilet construction.
So again, by looking at a policy in a number of
different contexts, you can learn a lot more about
what's driving the results.
I've got two minutes.
I haven't got on to my second example.
So let's see.
In Indonesia we found no impact on child height.
India also no impact on child height.
Mali, a large increase in child height, and we think
that was because in Mali we were starting from a base
where quite a few people already had toilets,
so when you had that big increase in toilet
construction you got close to the whole community
having toilets, and so everybody benefits.
Whereas in India, although you had, you know,
pretty high rates of toilet construction, starting from
such a low base, that you still ended up at the end
with a lot of people not having a toilet.
So even if you build a toilet yourself and your
neighbours don't build a toilet, then you probably
don't benefit that much in terms of health improvements.
Okay. Let me go through this quickly.
Maybe I'll just skip my second example and we can
talk about that in the questions if it's of
interest to anyone.
Scaling up.
So what was nice about this project was that we were
evaluating a program which was already being run at scale,
but more than that, the World Bank had trained
up two private organisations to implement
this program really kind of professionally and well,
and then they were scaling it up.
And so that organisation was then training local
government to implement the program.
And so within our treatment communities, approximately
half were being implemented by the World Bank team,
and approximately half were being implemented
by the local districts.
And so we can look to see what the impacts are when
it's implemented in a more kind of piloted environment
by the World Bank, versus when it's embedded in government,
which is nice, because that overcomes, you know,
one of the concerns about external validity
if you scale it up.
You know, is it still going to be effective?
And unfortunately in this case, all of the impact of
toilet construction and health was coming from
World Bank implementation.
So in fact the World Bank implementation program was
resulting in quite large increases in
toilet construction.
When you look over the whole sample, it kind of
gets washed out, because we were averaging over those
villages where it was implemented by local
government and villages where it was being
implemented by the Bank.
And when we looked at the local government
implementation by itself, we found no impact on
any indicator, no change in attitudes, no change in
tolerance of open defecation, no change in
toilet construction, no change in health outcomes.
So that was disappointing, but kind of interesting
from a methodological point of view in terms of,
you know, how you might want to structure an evaluation
so you can get at those questions.
I won't talk about that.
So I won't talk through this example, but just in case
anyone's interested I'm happy to talk about this later.
This was an evaluation where we facilitated the
provision of information about employment agencies
to women who were thinking of going overseas to work
in the Middle East or elsewhere in Asia.
And the idea was that these women - we did quite a bit
of research at the start - don't see the choice of
employment agency as being important, and they don't
hold the employment agency responsible in any way for
their experience in the destination country.
And so we did some research showing that you can
clearly discern which employment agencies are good,
because we interviewed women who returned.
We could see that the women who went to these better
agencies had better experiences overseas.
And so we provide information in these
facilitated sessions to women about which
employment agencies in their areas are reputable
on the basis of our research, and which ones
you really might want to think twice before going with.
And so now we're just in the process of following up.
So about a third of the people in our sample have
already moved overseas, so we'll be able to look at
which agency they went with, so whether our
intervention affected their choice, and then ultimately
what their experience was like in the destination country,
and so whether our intervention improved
outcomes for them.
And we've been working very closely with the Indonesian
government, and we're using some administrative data,
and they're interested in possibly scaling it up
if we do find impacts down the track.
Thank you.
(Applause)
Thank you very much Lisa.
I'm now going to call the whole panel up so we can
have a conversation.
So I'm going to be cheeky and take the first question,
but from then on it's yours.
So we're going to have two people with roving mics and
we're going to try and get to as many questions as we
possibly can, so keep those in mind.
So thank you very much all three of you for
a stimulating conversation, and there are many, many threads
I think we could pick up from each of your presentations.
But I thought I'd start with one of the most
obvious ones, which is in BX 2014 - four years ago
was the first of these sort of conferences,
and what was notable at the time is sort of how
geographically sort of concentrated a lot of the
behavioural insights work was.
And now of course Filippo your work, and the work of others,
has shown just how quickly it's sort of
penetrated different parts of the world.
And I'd like to start with a question which is
a bit two pronged.
First of all with Filippo, could you tell us a little
bit about what you found in your study about what
accounted for the geographic spread of
interest in behavioural insights in government?
You know, what do you think were the drivers of where
it went to, and first and fast?
And maybe also to Varun and to Lisa, in your experience
running RCTs in different parts of the world,
what's your experience dealing with different governments
about their receptiveness to being participants in
behaviourally motivated RCTs?
And do you think there's anything we can learn about
what makes those similar, and where there are kind of
opportunities for greater penetration, and why there
might not be as much interest in behaviourally
motivated RCTs?
So maybe I'll start with Filippo.
Yep, sure. Well, thank you Kate.
I think it's an excellent question.
I mean, what spurred the movement, if you like,
but then what also facilitated this predict?
Well I think probably one of the parts in the movement,
apart from the work which was already being done,
for example, by J-PAL in developing countries,
in emerging economies etcetera, was probably financial crisis.
That hit very badly rich countries, rich country economy.
Started in the US, then it spread to Europe.
Interestingly enough part of the world which used to
be hit by the financial crisis were not hit in that case,
of course Asia for example.
But I think that was interesting lessons,
and one was that in fact that economic crisis started as
a financial crisis, and that financial crisis
started in part because people were buying things
they didn't know anything about.
And even worse, those who are selling these financial
products didn't even know what they were putting
inside that financial product.
And then there was a strong push towards actually
making consumers more aware of what they were buying,
so information, and thinking about the risk of
not being salient or providing some sort of
information overload, providing, you know,
these are all the details, these are all the
term and conditions, just sign at the end.
In fact when we did the mapping for the 2017 -
I don't have the slide here, but we collected roughly
a hundred case studies.
The majority of case studies were coming from
the financial sector.
So these were regulators, regulatory agencies,
governments, in a number of OECD countries, which was
known as the rich country club, so the sample was
biased towards rich economies, which were
actually rushing to apply some of the behavioural
insights to address, if you like, what was a major failure,
and in part also motivated by the behavioural biases -
information, ... information etcetera.
That was probably this part in, if you like,
the rich countries, OECD member countries.
I think the map is showing that it's spreading,
it's spreading quite rapidly, institutionally but also
in terms of applications.
So I think we can safely say that it's not a fad.
It's clearly established institutionally,
it's established as a policy tool, and you saw example
from Lisa and Varun.
I guess thinking ahead is really,
what is the next step, right?
Probably even in terms of, you know,
sustainability going forward.
And here I'm thinking really - but Varun and Lisa
can probably speak more articulately about this -
is also the spreading, if you like, in countries,
emerging economies, developing countries,
where for example issues with the cost of experimentation
can be important, how you embed it in those situations.
So think about the cost of experimenting, but also
thinking about partnerships between government and
non-government institutions, academia,
but also strange beast.
I mean, you are becoming a bit of a strange beast,
or at least BIT has become a bit of a strange beast.
I mean it was inside government.
Now it's actually providing service and advice to other
governments from outside government.
There are a number of research institutions which
could work very closely with government.
I mean maybe they have lower academic standards.
I mean John List this morning was talking about,
you know, the high standard you should have for evidence.
Sure. But I mean for all those who work in government,
I think they know that, you know, you can't always have
the gold star.
So sometimes you need to have some compromises and
get the best you can get as quick as possible.
And I think having the support of non-government
institution outside government, which understand
the business of government, also in terms
of providing evidence, can be extremely important.
I think the third area to support the spread is
probably capability, how you build capability for
thinking about behavioural barriers within government,
not necessarily and not only within behavioural
insight institutions, but also in the analysts,
the economists, all those who work on policy every day.
I think these are probably some of the drivers for the ...
Fantastic. And Varun and Lisa, I'd love to hear sort of
receptiveness of developing country and emerging
economy governments, and third parties.
You know, do you see trends?
You know, do you find that you work in Peru and then
neighbouring countries are increasingly interested?
Or do you actually see that it has something to do with
the political makeup or democratic makeup of
those countries, or something else entirely?
Maybe Varun I'll start with you.
Yeah, that's an important issue.
A couple of thoughts, first on the various forms of the
use of behavioural insights, the work that the
OECD did on that slide.
A couple of colleagues are working on a sort of follow up
piece which is going to try and look at the
strengths and weaknesses and the preconditions of
different kinds of models.
You know, whether it's within government, or you know,
a strange beast, or something else, to sort of
like think through what institutionally is needed
and what the prerequisites are.
On the question of developing countries, I mean,
I think that some of the literature buffs might -
is it ... the person who said, oh, I've been speaking
prose all the time, I didn't know I was
speaking prose?
There's something like that, though our French
speaker might correct me.
But so, social norms have been a part of development
for a long time, simplification, you know,
has been a part of it for a long time.
The importance of context, the social scientists have
been doing it for a long time.
So in some sense developing countries, like, yeah,
they know that - for those of us who are economists,
who have come in the background with a neoclassical model,
it's a big change.
But if you're on, you know, the sort of the front end
of the policy design, people have been doing this
kind of thing and are aware of it, so I think it's not
necessarily such a big change.
I think the change is the experimentation side,
doing RCTs, you know, using evidence about what works,
and the process of adaptation, which I think
is actually a big struggle for people everywhere.
It's sort of like, you know, once you implement
a program and you get a result, then what does that
mean for how you redesign a program?
What does that feedback cycle look like?
I think that's the real challenge rather than,
you know, behavioural insights per se.
So, economists - I think one of the BX's -
I think Thaler said a little while ago, what's the future of
behavioural economics, and he said it's going to disappear.
But what he meant was it's just going to
become economics, right?
So it's coming through people, you know,
working in partnership with people, and ... governments
have studied recently, or you know, people from donor
agencies are sort of bringing that in.
So that doesn't seem like a hard process.
It's the evidence and the adaptation that I think are
the challenges and what are new.
And Lisa?
Yeah. So my experience is really just in one country,
Indonesia mainly, in terms of close collaboration
with government.
And so I've been working in Indonesia since 1994.
And so it's an interesting process to watch how
it's happened, and I'm not exactly sure how it's happened.
I think it's to do with people in key positions in
the bureaucracy in Indonesia, and also in the
Australian bureaucracy.
As I said, J-PAL was funded in Indonesia through DFAT.
There might be people in the room who know more
about this than me, but my observation was that there
was at least one key person who had influence and who
was very open to this kind of research, and he was
very effective in kind of collaborating with J-PAL,
getting J-PAL interested, and ultimately getting the
funds to flow to J-PAL.
But also within the Indonesian bureaucracy,
I think what you see is you have a lot of
high quality bureaucrats.
A lot have been educated in the US, and a lot of this
kind of work has come out of the US, so they're
a little familiar with it, and then they get into
a position where they could actually do something about it,
and they've taken that opportunity.
And often that's with J-PAL there, and so working very
closely in collaboration, and also with the World Bank.
That's certainly true in Indonesia with the World Bank
working very closely with J-PAL and the
National Planning Agency and other agencies in Indonesia.
So that's been really interesting to watch.
And I'm much newer to trying to do these kind of
things in the Australian context, but my observation
would be is that it's more difficult in the
Australian context.
That's how it appears to me.
Which might be to do with educational backgrounds.
So it's great to see everybody here.
And so just less exposure to these kind of ideas,
at least in the past.
Partly it's also to do with costs.
I mean, if you can use administrative data in the
Australian context, that's great.
If you have to collect survey data in a country
like Indonesia it's still pretty expensive,
but in Australia it's very expensive.
So there are those kind of issues too.
Fantastic. Okay. I'm going to open up to the audience
for questions.
And if you could just tell us your name, where you're
from when you start, that would be great.
Hi. I'm Deben Mitchell.
I'm from the Australian Financial Security Authority.
And Lisa, your presentation peaked my interest,
and I hear what you're trying to say about randomised
control trials, but please don't take this offensively.
But I was rather offended with some of the content of that,
where you actually talked about the nudge theory
of shaming people.
Okay? And this comes to the ethics of nudge theory.
Is that the right thing to do?
Have you thought about what should be the appropriate
nudge before just settling on let's shame people into
something and make them act?
And I say this because it's a generational issue.
It's gone on and on for a very long time, where the
developed nations think we know best of what's
required of the developing nations.
Like, if you think about it, the oldest
civilisations in the world were in the eastern part of
the world where they had more advanced sanitation
than anywhere else in the world.
And yet we now think that it's appropriate to shame people.
And I don't know how offending it might be for
the recipient of that message.
And as you say, the random control trial did end up
saying that no, shaming people didn't really help.
So I want to know from a practitioner perspective
what role do you play in making sure the ethics is right?
So that's a very interesting point,
because CLTS - because I share your concern about the
shaming aspect, so I'm not involved - CLTS is a whole
world of its own, right?
It was actually developed by someone in Bangladesh,
by a Bangladeshi in Bangladesh, so it's not
something that's imposed externally.
And when I bring up this concern with shaming,
people tell me that's because I'm western,
so I don't know where the truth lies there.
But, you know, I share that concern.
My role in the CLTS evaluation is as an evaluator.
So I was in no way involved in the design.
But nevertheless, CLTS is viewed very positively in
a lot of less developed countries around the world.
And so, you know, I guess the ethics of shaming, yeah,
is something that I think, you know, in the
policy implementation you want to look at that.
I was having a discussion with someone about this
recently and she pointed out that nobody's actually
established that it's the shaming that is resulting
in the increased toilet construction.
And so that's a very interesting observation,
because we do collect information on people's
attitudes to open defecation and whether that changes,
and we do see changes in that in the
World Bank implemented program, but it's pretty modest.
So even though it's been implemented widely -
so there's been RCTs now, but there's also been a lot of
qualitative research on it - I still don't think
people really have a good grasp of what the mechanism is,
and whether actually shaming is a necessary part
of that whole equation.
So I think that that certainly deserves more attention.
Varun?
Yeah. I mean my sense is that I think the walk of shame
may be unfortunately named.
Because, I mean, another way to think about what's
happening is it's making something salient,
because you don't necessarily know where contamination
is coming from.
But if you actually see it happen, it's just sort of
visible in a way.
In some CLTS communities there's also a joint
promise where you sort of raise your hand together,
and that's a different kind of mechanism.
It's not a shame mechanism.
It's sort of like the community uplifting itself mechanism.
So that's part of the story.
And then I also think that the shaming - you know,
in a couple of states in India, in Harian I believe,
there was a law passed that said you couldn't run for
the village panchayat unless you owned a toilet.
It was like a marker of, like, status, you know.
And in our sample, you know, in some places
in the villages, there's these kids that run around and
they smack people with sticks, you know,
who are like out there and defecating.
I mean I share the concern.
When I first heard this I thought wow, you know,
but at the same time I think that we all shame and it's
sort of like it's part of the process that's
happening almost organically anyway.
And I might just add something, because I've
just had another thought.
So the extra funding that we got to look at this program,
CLTS, enabled us to collect information on social capital.
And so then we used that social capital data to try
and understand the mechanisms.
And so we did observe that in villages where they had
high levels of social capital prior to the
implementation of the program, you got much
better results, and actually got very poor
results where you had low levels of social capital.
And then we tried to understand why that was.
So we collected information on how information was
transmitted within the village, and the
willingness of people to contribute to the public good.
Because we thought if you're in a high social
capital environment, maybe that's what driving it,
but we didn't find any difference.
What we found was that in a high social capital
environment shaming seemed to work better,
because social sanctions were being used to motivate people
to build toilets.
And so we found that social sanctions were much more
effective when you had high levels of social capital,
presumably because when you're in a high social
capital environment people care about what other
people think of them, and so the program was more
effective in that environment.
Kate, can I quickly - just that I find it extremely interesting.
I'm going back also to the work we are trying to
develop on this policy tool kit, behavioural insights
policy tool kit and ethical framework.
I think these are interesting question,
and probably is the kind of question someone in
government would ask himself or herself before
actually calling Lisa or Varun, and start thinking
about behavioural insights.
And this is about the guidance we were thinking about,
really for policymakers, to actually ask yourself up front
some of the questions you want to ask yourselves about,
you know, possible behavioural applications,
and when do you want to ask these questions and what
kind of questions you want to ask, before you call the
... who can actually implement a nudge etcetera.
So that was a bit of thinking, also mainstreaming, right?
And so making sure that, you know - yeah, shaming,
but it's also, you know, social norms and how you
use social norms.
And that mainstreaming should be in government,
probably even before you think of having a
behavioural insight intervention.
Right. I think it touches on a point that many people
have made, which is where does the limit of the
policymaker and the politician begin and end.
So a lot of people would say that actually there's
a democratic process out there helping us to best
understand what the kind of values of our kind of
entity might be, and then policymakers are involved in
that process but they don't dictate all of those values.
And therefore there's some things which need to
necessarily go up one level to know is this something
we're going to be comfortable doing?
If we found out that this was an effective technique,
would we be comfortable rolling out this particular
technique across the economy?
Because those two things may not be one and the same,
so that kind of constant dialogue between policymakers,
evaluators, and the kind of political sphere
is also kind of an important technique there I think.
I think we had another question just in the same
sort of middle section.
Thank you, it was very interesting.
I'm Jessie Detal from Department of Human Services.
There's been mention about, you know, a lot of studies
are on the WEIRD population versus there's not enough
on non-WEIRD populations, so that's great to see all
the studies done that you've talked about today.
So I'm just curious.
I mean, you know, one of the criticisms of
behavioural insights or behavioural economics is
there's this big long string of behavioural
biases and so what, and it can be confusing and one
bias can contradict another one.
So I'm just wondering, based on your experiences
working in the non-WEIRD populations, are there
particular behavioural biases that either
stand out as, you know, really consistent, you know,
because we're just all human beings at the end of the day?
Or conversely, are there some that are really quite
culturally specific?
Excellent question.
I think everyone's going to have something to say about that,
so we might start with Lisa and then head across.
Well I'll just talk about my personal experience.
So I started off as a PhD student running my first
experiment in Indonesia, and I was looking at the
use of money to incentivise behaviour.
And so a criticism of experimental economics at
that stage was that, you know, we play all these
games to elicit people's preferences - but it's
normally for $20 or something, it's not a lot
of money - so maybe what you're observing isn't actually
real world, maybe you have to play with more money.
So I went to Indonesia and I played a range of games with -
I don't know if you'd call them a WEIRD population.
It was a population of students at a university,
so it's not typical of the Indonesian population,
but where I played for small amounts, so similar in
purchasing power to what you'd see that had been
playing into Europe and the US, and elsewhere.
This was an experiment that sought to elicit altruism,
and we saw very similar results, incredibly similar
actually in the Indonesian population.
And then we raised the stakes madly and we saw
that the results didn't change all that much.
I've also run those kind of experiments in China and
done comparisons with Australia, and you do see,
particularly with social preferences, you do see
quite marked differences.
So we were looking at acculturation, so looking
at how people's preferences changed.
We were looking at people from China migrating to
Australia and, you know, how their preferences
changed the longer they'd been in Australia,
and we did see people's preferences changing to
becoming more like Australians, which was very
bad news, because that meant they were becoming
less altruistic, less trusting, less trustworthy,
more competitive.
Well is that good or bad? I don't know.
But you know, so there were distinct cultural
differences there.
And just briefly, another example is we ran
corruption games in Singapore, India, Indonesia
and Australia looking for differences in attitudes
to corruption, and we did see stark differences in the
way people played these games across the countries.
Maybe not in the way you'd expect.
But one thing that was really interesting was we
saw gender differences.
So there's large literature showing that women are
less likely to engage in corruption than men,
so push for more women in government and so forth.
But most of that literature comes from countries
like Australia, the US, Europe, right?
And so we saw that clearly in the Australian data,
but when we looked at Singapore, India and Indonesia,
there were no gender differences.
So that was quite interesting, yeah.
Great. Filippo?
Maybe two points there.
One is about culture and context, and whether,
you know, something which works in a particular context can
work in another context.
And maybe also something on the WEIRD population,
which we can also read it as using sort of lab
experiments or, you know, a group of students in a
control environment to see actually how they react to
a particular intervention.
On culture, well, I think as Varun was saying,
the whole spirit, I would say, of behavioural insights
is experiment, and having an inductive rather than
deductive approach.
So essentially instead of assuming a behaviour,
perfect rationality etcetera, you just go and find out.
And I think there is evidence that context matters.
I think it was a BIT quite famous experiment where you,
you know, tax form, you ask people to sign at
the beginning of the tax form then they fill in the form.
And I think the experiment was quite conclusive in the UK,
the number of people make mistakes when filling
the tax form goes down quite significantly.
I think they tried the same experiment in a country
which I won't name, but of which I am a citizen,
and you can easily understand given my name,
Filippo Cavassini, and I can also assist on my accent
if you like. Well, it didn't work.
But as you say about culture in my home country, I don't know.
Maybe that, you know, making a commitment doesn't
then lead through when you make it at the beginning
and then at the end of the tax form.
But so context matters, hence the importance of
testing and not simply copy and paste.
In terms of WEIRD population, I mean, I think
there is quite good evidence that for certain issues,
problems, etcetera, I think having policy labs
where you have sort of a group of people just
replicating the real world without doing the
randomised control trial can be a cost effective way
of testing a possible intervention, so it could work.
Is it WEIRD because it's a group of students?
Again, I think also there context might matter.
For the work we are doing in Scotland, for example,
we will be running together with ... with Dr Peter ...,
we run some policy experiments, policy labs.
And there were interesting discussions with the
Scots actually, on whether some of the labs could be run in
Ireland where ... is based, rather than Scotland.
And the Scots were quite adamant in saying that
no, no, no, no, actually the labs should be in Scotland
because even if it's in a control environment,
you know, we want a Scottish control environment. So be it.
I think it's important to take into consideration
context also when you have WEIRD population, but this should
not prevent you from using sometimes WEIRD population.
Varun?
Yeah. So I mean I think that there is a fair amount
that's universal that is useful to think about.
I think there's two key ideas.
One is that thinking hard consumes biological energy,
right?
There's a 1974 Kahneman paper I like a lot in which
people are sort of shown a four digit number and
they're asked to add three, number three, each of the
four digits, and do this repeatedly.
And after a short period of time people's pupils dilate
50 percent, their heart beats go up seven beats a minute.
It's just to think hard actually consumes
biological energy, and that's why to give attention
to something, it's just human to sort of feel tired.
And that might be, as Cass was saying yesterday,
behind several of these biases, you know, anchoring,
loss aversion, they might be related to sort of the
attention and energy required to actually shift,
and I think that's pretty universal.
And the other part of it is we're all
social animals everywhere, right?
So we have families, we have, you know, groups,
and we have in groups and out groups.
We all have social norms.
What the content of the norm is, that varies,
but that we have social norms, that's pretty clearly universal.
So that's the sort of larger thing.
I think, you know, yes, the sort of the program of
focusing on these things, I'm pretty confident.
And then in terms of, like, what to focus on,
and I think John List has this - I'm not sure he said it
this morning exactly - but sometimes we overemphasise
the population at the expense of the situation.
And I think that generally speaking organisational
variation might be more important than
population variation, or even cultural variation.
Because sort of like the situation that you're in
and the kind of social practices in the organisation,
they really matter.
I don't know if you know some of the work on
altruistic punishment.
You know, and in some lab studies people play these
public goods games and then other people watch,
and they can punish people who don't, you know, cooperate.
I think Benedict ... did this.
They find that in many countries people do punish
people who are free riders, but in a handful of
countries with high levels of corruption,
they actually punish the co-operators.
Then you're incurring the obligation on me,
and so you don't really want to be a co-operator,
because that's basically making someone dependent
on you for the future.
And that's an organisational thing,
you know, that's not a human thing.
It's sort of like some societies have a different
understanding of what law means and what dependence means.
But I wouldn't put that on culture, I'd put that on
organisational structure, generally speaking.
So we've just run out of time, sadly, because I
think there's probably loads more conversations
that could take place.
But I think this kind of general notion of sort of
the distinction that we often forget about
cognitive neuroscience and cognitive psychology versus
social psychology, we tend to blur, you know,
with this overarching term about behavioural insights.
Sometimes it makes it harder to identify the sort
of specific threads that we actually want to draw upon.
So I hope you've all found that you've learnt things
you didn't otherwise know.
Thank you very much to our three speakers,
and if you'll join with me in saying thank you.
(Applause)
