"The" Social Credit System
Toni
The Chinese Social Credit System (SCS) has been discussed a lot in Western media. However, we do not know currently how the system that is supposed to take nationwide effect by 2020 will look like, as there are more than 70 pilot projects currently undertaken. These pilots rank from commercial royalty and rewards programs (Sesame Credit) to an Orwellian system, where each action has a predetermined associated score (Rongcheng). In-between, there’s nebulous algorithmic systems that basically act as a Black Box (Honesty Shanghai). This talk, therefore, looks at some of these pilots and their implementation details, and through an agent-based modeling framework, discusses the likely effects of different implementations. In doing so, it shows that most of the systems currently being tested are prone to manipulation by leaders from all levels of government, and that the ostensible goal of allocating scarce resources more efficiently is unlikely to be served by the new system(s).
 Alright, then it's my great pleasure to introduce Toni to you. She's going to talk about the social credit system, which kind of feels to me like a Black Mirror episode coming to life. So, I'm slightly nervous and really curious what we're going to learn today. So please give a huge warm round of applause and welcome Toni. So, good morning everyone. Before I'm going to be talking, going into my talk, I'm just going to be presenting the Chinese translation streams for everyone who just doesn't speak English. So, we're going to be talking about the social credit system in China. Where the social credit system that you always hear about in Western media actually doesn't really exist. And most of my talk will actually be talking about what all we don't know, which could fill an entire hour or even more. But I'm just going to be focusing on some of the most interesting things for me. First of all, a little bit about me. I'm an economist, but I'm not only concerned with money. I'm kind of looking at economics as the study of incentives, which means that what I'm really interested in is how humans respond to different kind of incentives. I don't believe that humans are completely rational, but I do believe that humans do try to maximize what they think is their best interest. Now, some words about me. I studied math, economics, and political science in a couple of different cities all around the world. I spent overall 19 months in China. Most recently, I was there in July on a government scholarship, which was really, really interesting because while there, I read all of these Western newspaper articles about the Chinese social credit system. And I went to a pretty good university and I asked them, so what do you think about this system? And most of them basically looked at me blindly and were like, what system? I haven't even heard of this. So that was kind of an interesting experience to me because in the West, it's like this huge all-encompassing system. And in China, most people that aren't directly in touch with it actually don't know anything about this. I'm broadly interested in the impact of technology and society, life, and the economy, obviously. And in my free time, I do a lot of data science and machine learning with Python and R. So I thought it was quite interesting to look at the social credit system also from this point of view because you always heard that it's like this big data initiative. And then when coming down to it, what you actually see is that they don't actually use machine learning all that much. They have basically a rule-based catalog where if you do this, you get 50 points. If you do this, you get 50 points. And then they actually have a lot of people that are reporting on other people's behavior. I'm going to be talking about how exactly it looks later on. But I was very, very surprised after reading a lot of the Western newspaper articles that were basically, oh, this is this big dystopia or well-yain with big data working. And then you read what's actually happening. And they have huge lists of if you, J-Wog, you get 10 points distracted from you, this kind of thing. If you want to get in touch with me, you can use Twitter, but you can also use different emails, either my professional email or my personal email address that you can both see there. If you have any thoughts on that or are interested in this a little more, I can give you more resources as well because obviously today's talk will only be scratching on the surface. So perceptions of the social credit system. One of the interesting things that I've talked about before was how in the West and in China, the perception is completely different. So in the West, which is from FinancialTimes.com, you see this huge overwhelming guy and he basically puts every Chinese person under a microscope. They're all kind of hunched over and everyone has this score attached to them and they seem pretty sad and like very, very Orwellian concept. Whereas in China, this is actually from a Chinese state media. And what it says is, well, we can all live in harmony with this new system and all trust each other. And interestingly, Chinese people actually believe that to some degree. They believe that technology will fix all this current problems in the in society, especially because in China currently trust is a rare commodity. And this new system will lead to more efficiency and trust and a better life. And I have a really, really interesting quote from a Western scholar that really summarizes the Western perspective. What China is doing here is selectively reading its population to select against that, the trade of critical independent thinking. This may not be the purpose. Indeed, I doubt it's the primary purpose, but it's nevertheless the effect of giving only obedient people the social ability to have children, not to mention successful children. This basically plays with the idea that if you have a low score currently in the cities that are already testing this system, what happens is your children can't attend good schools. What happens is you cannot take trains, you cannot take planes, you cannot book good hotels. Your life is just very, very inconvenient. And this is by design. This is kind of the plan. The Chinese government, they say it's a little different. The idea is about changing people's conduct by ensuring they are closely associated with it. One of the main things about this system is there isn't very much new data being generated for the system. Instead, what's happening is all the existing data that is already collected about you is basically combined into one big database for each and every person by your ID number. So in China, once you're born, you get an ID number, which is similar to a social security number in the US. We don't really have a similar concept in Germany. And it used to be that your ID number was only necessary for public, like for government stuff, but now you need your ID number for getting a bank account. You need your ID number for buying a cell phone. Even if it's a prepaid cell phone, you still need your ID number. So all your online activity that happens with your cell phone is associated with your ID number, which means you can't really do anything anonymously because it's all going back to your ID number. There's a couple of predecessors, some of them going actually back to the 1990s, that are supposed to be integrated into the new system. One of them or like two of them are blacklists. One of them is a court blacklist. So in China, courts work a little bit differently. They tend to like giving you fines as they do in other countries, but they also like giving you apologies to do. So one of the things, if you do something, for example, you're a company, your food safety wasn't up to par, you have to pay a fine. But in addition to this fine, you also have to write a public apology letter in the newspaper. How you're very sorry that this happened and it won't happen again. And it was a moral failing on your part and it won't happen again. And if you don't do that, you go on this blacklist. Similarly, if you take out a line of credit and don't pay it back within three months or don't do any payments for three months, you go on this debtor's blacklist. If you're on this blacklist, which again is associated with your Shenfeng drone, so your ID number, what happens is you cannot take trains, you cannot take planes. Your life basically becomes very, very inconvenient. Your children can't go to good public schools, your children can't go to private schools, your children can't go to universities. All of these issues are suddenly coming up. There is also a company database that's called Credit China, which is basically similar to the public debtor's blacklist, but it's basically a credit system, a credit score for companies. And then there's the credit reference center of the People's Bank of China, which is a credit score. It was supposed to be like Shufa or like the US FICO for individuals. But one of the big problems in China is that there are a lot of people that aren't part of the formal economy. A lot of people are migrant workers. They get their money in cash. They do not have bank accounts. They do not have anything, they do not have rent or utilities or anything like this because they live in the country, so they own their own home, which they built themselves so they didn't even finance it. And their home isn't officially theirs because in China you can't actually own property. Instead, the government leases it to you. So there were a lot of people that were not covered in this system. And I think the last data that I had was that less than 10% of Chinese adult citizens were actually in the system and had any sort of exposure to banks, which is very, very little. And that meant that people couldn't get credit because banks would only give credit to people that were in the system or people where they had some sort of handling on whether they would be paid back. Now, the implementation details of the new system are very, very scarce. But the basic idea is that Chinese citizens are divided into trustworthy individuals and what the Chinese call trust breakers. Sometimes you have five different groups, sometimes you have two different groups, but in general there's sort of this cut off above this line, it's good and beyond this line, it's bad. This is one graphic from the Wall Street Journal that just shows some of the inputs that go into the system. And one of the things that we see is that the inputs are crazy, crazy varied. So it is, do you pay income taxes? Do you pay your utility bills on time? Do you respect your parents? However they measure that. Do you have a criminal record? Do you pay for public transportation or have you been caught not paying? What about your friends? Do you retweet or use WeChat to distribute sort of information against the party which they call reliability? In actuality it's not about whether it's factual, it's about whether it's against the party or not. Where do you buy and what do you buy? Apparently if you buy diapers it's better than if you buy video games for your score because you know if you buy video games obviously you're not very responsible and if you buy diapers you have a kid you're sort of conforming to the societal idea. And then your score is supposed to go into all these different categories. You're supposed to have better access to social services if your score is good. You're supposed to have better access to internet services. So in theory the idea is that at one point if your score is too bad you're not allowed to use WeChat anymore. You're not allowed to use Alibaba anymore. You can't become a government worker. You cannot take planes and high-speed trains. You cannot get a passport and your insurance premiums will go up. So it's supposed to be this really really big overwhelming system but in actuality what they say their stated goals are is it's a shorthand for a broad range of efforts to improve market security and public safety by increasing integrity and mutual trust in society. So one idea is to allocate resources more efficiently. Resource allocation in China is a pretty big problem because people grow up with there's 1.3 billion people so there's it's always going to be scars and a lot of stuff is people grow up with this idea that it's just very very scars and current distribution strategies which are mostly financially based but also often one-T based don't really seem fair. For example public transport in China is highly subsidized which means that the price does not reflect whether does not reflect true scarcity. So currently the way it works is in theory it's first come first serve and practice there's people that are buying up all the tickets for for example the high-speed train from Shanghai to Beijing and then selling it at a profit or selling it to certain companies that have good ties to the government. That seems very unfair so the new system is supposed to distribute them more fairly and more efficiently. The other thing is restoring trust in people. Perceived interpersonal trust and trust in institutions is extremely low in China. If you're from Germany you might have heard that there's Chinese gangs basically buying up German milk powder and selling it in China. This is actually happening because in 2008 there was a big scandal with laced milk powder and ever since then anyone who can afford it does not use Chinese milk powder because they don't trust the government or the regulations the firms enough to buy Chinese milk powder. So they're actually importing this and the the big irony is sometimes this milk powder is produced in China exported to Germany and then exported back to China. The social credit system is then supposed to identify those that deserve the trust and the third point is sort of a reeducation of people. The idea is they want to make people in the image that the Communist Party thinks people should be and one additional way to the punishments and rewards this could work is the feeling of being surveyed because you can't do anything anonymously. You will automatically adapt your behavior because you know someone is watching you all the time and this is how a lot of the Chinese firewall actually works because most people I know that are sort of more more educated they know ways to circumvent the Chinese firewall but they also know that they're always being watched so they don't do that because you know they're being watched so they self they censor ship they censor themselves. As I've said before allocation of scarce resources so far is mainly through financial or guanxi channels. Guanxi is basically an all-permeeting network of relationships with a clear status hierarchy. So if I attend a school everyone who also attended this school will be sort of in my guanxi network and there's this idea that we will have a system where we are all in group and in group we trust each other and we do favors for each other and everyone who's outside of my immediate group doesn't I don't trust and I don't do favors for and in some ways the guanxi system right now is a substitute for formal institutions in China. For example if you want a passport right now you can of course apply for a passport through regular channels which might take months and months or you can apply for a passport through knowing someone and knowing someone which might take only two days whereas in Germany you have these very regular formal institutions in China they still use guanxi but increasingly especially young people find that guanxi are very unfair because a lot of these are who are where you went to school which is determined by where you're born who your parents are and all these things. Another thing that's important to understand because the system works through public shaming and in a lot of western society we can't really imagine that like I wouldn't really care if my name was in a newspaper of someone who jaywalked for example it would be oh well that's okay but in China this is actually a very very serious thing so saving face is very very important in China and when I went to school there I actually we had this dormitory and it was a all foreigners dormitory where the staff that were responsible for the dormitory felt that foreigners were not behaving in the way they should so their idea was to put the names the pictures and the offenses of the foreigners in the elevator to shame them publicly so for example if you brought a person of the opposite sex to your room they would put your name your offense and your room number in the elevator and of course this didn't work because for a lot of western people it was basically like oh well I'm going to try to be there as often as possible because this is like a batch of honor for me and the Chinese people they figured well this is really really shame and I'm losing my face she brought alcohol so this didn't really work at all but this is kind of the mindset that is behind a lot of these initiatives as I've said there's a lot of problems with we don't really know what's going to happen and one of the ways that we can see what might happen is actually to look at pilot systems China has or like the ever since the communist party took hold that the Chinese government has tried a lot of policy experimentation so whenever they try a new policy they don't roll it out all over but they choose different pilot cities or pilot districts and then they choose oh well this is the district where I'm going to be trying this system and I'm going to be trying another system in another district or city and this is also what they did for the what they're doing for the social credit system now I have three systems that I looked at intensively for this presentation overall there's about 70 that I know of the swimming system swimming is a city in in China the wrong trunk system another city in China and season credit season credit is a commercial system from Alibaba I assume everyone knows Alibaba they're basically the Chinese Amazon except they're bigger and have more users and make more money actually and they have their own little system one of the problems with this kind of system that I found when I tried modeling it was that it's a very very complex system and small changes in input actually changed the output significantly so when they try usually when they try this pilot system they basically have a couple of pilots then they choose the pilot that is best and they roll it out all over but for this kind of thing where you have a lot of complex issues it might not be the best way to do that the swimming system is actually considered the predecessor of all current systems it had a focus on punishment and it was quite interesting at the beginning of the trial period they published a catalog of scores and consequences here is an example this is basically taken from this catalog so if you took out bank loans and didn't repay them you've got deducted 50 points everyone started with a thousand points for this system if you didn't pay pay back your credit cards you also got deducted 50 points if you evaded taxes also 50 points if you sold fake goods 35 points were deducted and actually the system was abolished I think in 2015 2016 because all the the Chinese state media and also a lot of internet citizens talked about how it's an Awellian system and how it's not a good system because it's all very centralized and everything that you do is basically recorded centrally but Krimers writes nonetheless the swimming system already contained the embryonic forms of several elements of subsequent social credit initiatives the notion of dispersional disincentives against rule breaking public naming and shaming of wrongdoers and most importantly the expansion of credit mechanism outside of the market economic context also encompassing compliance with administrative regulation and urban management rules so one of the things that is difficult for especially German speakers is that credit in Chinese Xinyou means credit as in loan but also means credit as in trust so the social credit system is one way of trying to conflict those two the economic credit and the trust credit into one big system but the swimming system basically failed so they adapted the system and are now practicing a new kind of system the wrong term system whenever you read a newspaper article on the social credit system in the west most people went to wrong term because they just received a couple of awards from the Chinese government for being so advanced at this social credit thing but it's it's very difficult to call this one system because there's actually many many intertwined system there is one city level system where city level offenses are recorded for example tax evasion and there's a couple of there's a couple of rules if you evade taxes your score goes down 50 but then if you live in one neighborhood your score might go up for volunteering with the elderly if you live in another neighborhood your score might go up for for example planting some trees in your garden or backyard so depending on your name of your score might be different if you work for a if you work for a taxi cab company for example they also have their own little score system and your score might go up if you get good reviews from your drive from your passengers your score might go down if you don't follow traffic rules these kinds of things there are and there are designated scorekeepers at each level so each district chooses a couple of people who are responsible for passing on the information to the next higher level about who did what there is supposed to supposed to be an official appeals procedure so whenever your score changes you're supposed to be notified but apparently that's not happening at this point for most people um again it's a system of data sharing and one thing that they haven't really disclosed yet is what kind of data is shared are they only sharing the points so if i'm in a district and i plant some trees does the central system get the information person a planted some trees or does the central system get the information person a got five points we don't know at this point and it would mean something very different for how the system could be used but still the end result at this point is that there's one score so you have one central score and it's kind of there's all these different smaller systems that go into this score but at the end everyone has one central score and currently about 85 percent of people are between 950 and 1050 so you start up you start off with 1000 and those are basically the normal people and then anyone above 1050 is considered a trust a trustworthy person and anyone below 1050 is considered a trust breaker and as i've said before with the naming and shaming and all these things what you can actually see here is a billboard with the best trustworthy families in wrong throne so these are the families that have the highest scores for example season credit is a little different it's the only system that actually uses machine learning and artificial intelligence to determine the outputs in wrong chung for example they have artificial intelligence they have computer vision for the most part and the computer vision cameras they decide they try to recognize you when you jaywalk and then when they recognize you when jaywalking you get a small sms well we just saw you jaywalking your score is now dropping but how far how the score develops depending on your jaywalking isn't really determined by machine learning or artificial intelligence instead it's determined by rules you know well one time jaywalking deducts five points and this is stated somewhere um season credit doesn't work like that instead it uses a secret algorithm and the way i talked to some people that work for season credit or for alibaba and the way they described it was they basically clustered their clustered people based on behavior then gave scores to these clusters and then afterwards um did basically reverse engineered their own score using machine learning so that whenever something new happens you can move to a different cluster um this season credit was actually refused accreditation as a as a credit score um in 2017 so banks are not allowed to use the season credit score for your um to use the season credit score to determine whether they give you loans or not because season credit is quite ingenious obviously alibaba wants to keep you within their platform so if you buy using alibaba and using alipay your score goes up if you buy using wechat pay which is a competing platform your score goes down and this uses many of the same rewards mechanism of the official government systems and this is just an illustration of what kind of scores you can have apparently your scores can go between 350 and 850 and in chinese there's basically five different levels so 385 is um trust breaker or missing trust and then 731 is trust break trust is exceedingly high so um one way i tried to approach this issue was through agent-based modeling social credit system is individual level but what we are really interested in or what i'm really interested in is actually societal level consequences so if everyone gets this score what does that mean for society um agent-based modeling works quite well for that because it allows us to imbue agents with some sort of rationality but with a bounded rationality what does bounded rationality mean usually in economics people assume agents are completely rational so they are profit maximizers they have all the information but in reality agents don't have all the informations they have a lot of issues with um issues with keeping stuff in their mind so a lot of the time they won't choose the best thing in the world but they choose the best thing that they see um and bounded rationality allows us to account for this thing it allows us to account for um heuristics and these things and what i did is i took the propensity for specific behavior from current state of the art research mostly from behavioral economics for example um i looked at tax evasion and i looked at who is likely to evade taxes in a in a system and then obviously there was some stochastic some chance element but the distribution that i chose is um related to the current research and i also checked that um my model has similar results to the wrong-tongue model which i modeled at at the beginning um so on average 87 percent um of my users have a score of within 10 percent of the original score which is also the data that um wrong-tongue city actually publishes now um for the most part i compared four different i compared design choices in two axes one of them was a centralized system versus a multi-level system and a rule-based system versus versus a machine learning system the centralized system is basically you have a central you have um all the information is kept centrally and um everyone in china or wherever in wrong-tongue has the exact same scoring opportunities um now if you have a centralized system the clear expectations were um were pretty good but at the same time the acceptance from the population was really really low um which they found during the swimming experiment and there's also the problem of a single point of failure who decides the central catalog and um depending on who sort of has the power it kind of just um reproduces power structures so because you have this central catalog the people that are in power centrally they are basically deciding some sort of score mechanism that works for them very well so that they and their family will have high scores a multi-level system um has the advantage that local adaptation kind of works and there's sort of many points of failure but um in my model when I allowed locals to basically set their own rules what happened was that they competed so it started out being this district of wrong-tongue for example and this district of wrong-tongue they compete for the best people that they want to attract and suddenly you have this kind of race to the bottom where um people want to move where they wouldn't be prosecuted so they move to places where there's less cameras for example at the same time there's many points of failures especially the way it's currently set up with um people reporting data to the next to the next high level and a lot of the time what we have actually seen in wrong-tongue was that they reported data on people they didn't like more than data on people they did like or their families got better scores than people they didn't know so it also kind of um reproduced these biases the um rule-based system has the advantage that um people were more prone to adapt their behavior because they actually knew what they needed to do in order to adapt their behavior but the score didn't really correlate with the important characteristics that they actually cared about and um as opposed to if in this machine learning system you know how in Germany we don't really know the shufa algorithm and I for example don't exactly know what I could do in order to improve my shufa score and this is a similar system in China with the sesam credit score a lot of people don't really they they say well I really want to adapt my behavior to the score to improve my score but when I tried doing that my behavior my score actually got worse and you can have different biases that I'm going to be talking about in a little bit um there's also this big problem of incentive mismatch so um the decentralized rules-based systems like rongchong which is the system that I analyze the most why because I believe this is the system that we're moving towards right now because rongchong won a lot of awards so the Chinese government the way they usually work is they try pilots then they choose the best couple of systems they give them awards and then they they roll out the system nationwide so I assume that the system that's going to be the system in the end will be similar to the rongchong system now one problem that I actually saw in my um simulation was that you could have this possible race to the bottom um there's also this conflict of interest in those that set the rules because a lot of the time the way it works is you have come you have your company in your company you in in combination with your um party leaders actually decide on the rules for your um for the score system but the scores of all your employees actually determines your company's score if you employ a lot of people with high scores you get a better score so you have this incentive to give out high scores and to make sure that everyone gets high scores but at the same time the government has an incentive for scores to be comparable um so there's a lot of int incentives mismatch the government also has the incentive to keep false negatives down but they actually the way the Chinese system currently works is um they emphasize catching trust breakers more than rewarding trust follow or like trustworthy people so um false positives for them are less important but false positives erode the trust in the system and they lead to a lot less behavioral adaptation I was actually able to show this using some um nudging research where that showed that as soon as you introduce a couple an error probability and you can be caught for something that you didn't do your your probability of um changing your behavior based on the score are actually is actually lower and in Rongchang one of the perverse things that they're doing is you can um you can donate money to the party or to party to like party affiliated um social services and this will give you points um which is kind of an indulgence system which is quite interesting especially because a lot of these donation systems work that way work in a way that you can donate 50 000 renminbi and you get 50 points and then you donate another 50 000 renminbi and you get another 50 points so you can basically donate a lot of money and then behave however you want and still get a good score and um the trust in other people can actually go down even more in the system because suddenly you only trust them because of their scores and um the current system is set up so that you can actually look up scores of everyone that you want to work with and if they don't have a score high enough then suddenly you don't want to work with them um the trust in the legal system can also decrease actually um why because trust in the legal system in china is already low and a lot of the things like jaywalking they're already illegal in china as they are here but no one cares and suddenly you have this parallel system that um punishes you for whatever but why do you why don't you just try to fix the legal system which would be my approach you can suddenly illegal activity could happen more offline and this is one of those things that um is quite interesting in countries that we've seen that have moved towards mobile payments and away from cash you see less robberies but so you don't see less um you don't actually see less crime instead you see more new types of crime so you see more credit card fraud you see more um phone robberies these kinds of things and this is also where um things could move for um that in the chinese case one major problem is also that this new system i've talked a little bit about this one but it can introduce a lot of new bias and reproduce the bias even more so um for example um china is a country of 55 minorities the hun are a big majority they have about 94 percent of the population so any computer vision task we've shown that they are really really bad at discriminating or at discriminating between individuals in um smaller ethnic groups in the u.s most computer vision tasks perform worse for african americans they perform worse for women because all of the training sets are male and white and maybe asian in china all of these tasks are actually um performing worse for ethnic minorities for the yuigurs for example and one way that they could try to abuse the system is to basically just what they also what they're also doing already in qinjiang is to basically just identify oh this is a person of the minority well i'm just gonna go i'm just gonna go and check him or her more more thoroughly this is actually what happens in qinjiang if you're in qinjiang and you look like a turkish person or like from turkish people um you can you are a lot more likely to be questions you're a lot more likely to be stopped and they ask you or require you to download spyware on your phone um and this is currently what happens and this new kind of system can actually help you with that um i've said that it can reproduce these kind of power structures and now obviously we all know newtville technology doesn't really exist but in the chinese case in the social credit case they don't even pretend they always say well this is newtville technology and it's all a lot better but actually it's the people currently in power they decide on what gives what gives you point and what detect what deducts points for you at the problem currently the the entire system is set up in a way that it's all goes together with your shan fendrong with your id card what if you don't have an id card that's foreigners for one but it's also people in china that were born during the one child policy and were not registered there's quite a lot of them actually they're not registered anywhere and suddenly they can't do anything because they don't have a score they can't get a phone they can't do anything really and um part of the part of the push with this um social credit system is to go away from cash actually so if you you need to use your phone to pay but for your phone you need your shan fendrong if you don't have a shan fendrong well tough luck for you and um currently the system in wrong trung is set up in a way that you can check other people's scores and you can also see what they lose points for so you can actually sort of choose to discriminate against people that are gay for example because they might have lost points for going to a gay bar which you can lose points for another big issue currently is data privacy and security um personal data is grossly undervalued in china if you ask a chinese person what do you think how much is your data worth they're going to be saying what data i don't have data um and currently the way it works is if you have someone's id number which is quite easy to find out um you can actually buy access to a lot of personal information for a small fee so you pay about a hundred euros and you get um all hotel bookings of the last year you get information of who booked these hotels with them you get information of where they stayed you get trained bookings you get access to all of the official databases for this one person and for another 700 rendman b you can actually get live location data so you can get the data of where this person is right now or where he's on her phone is right now but if you've ever been to china you know that where the phone is usually the people aren't far um soup china actually did an experiment where a couple of journalists tried buying that because it's actually these kind of services are offered on we chant pretty publicly and you can just buy them quite easily um so one additional thing that i looked at is um because one of the things that is quite interesting is you have this idea of credit as twofold credit is um trust credit but credit is also low in credit and what if credit institutions actually use this unified credit score to determine credit distribution the idea is that it's supposed to lead to reduced information asymmetry obviously so fewer defaults and overall more credit creation new people are supposed to get access to credit and they're supposed to be less shadow banking but what actually happens i'm not going to be talking about how i set up the model but just about my results if you have this kind of score that includes credit information but also includes morally good or measures of being morally good what you have is in the beginning about 30 more agents get access to credit and especially people that previously have not gotten credit access suddenly have credit access but the problem is that this social credit score that correlates all of these different issues um it correlates only very very weakly with repayment ability or repayment wishes and thus suddenly you have all of these non-performing loans um you have and what we see is sort of like we have non-performing loans banks give out less loans because they have so many non-performing loans and um then the non-performing loans are written off and suddenly banks give out more loans but you have this oscillating financial system where you have where you give out a lot of loans a lot of them are non-performing then you give out a lot of loans again and this is very very vulnerable to crisis if you have a real economic crisis during the time where non-performing loans are high then a lot of banks will actually default which is very very dangerous for a financial system as nationed as the Chinese one now what are some possible corrections you could create a score that um basically is the same as a shoofa score so that looks only at credit decisions but suddenly you lose a lot of incentives for the social credit score if the social credit score doesn't matter for credit distribution anymore another ish thing and this is i think the more likely one is that you have a blacklist for people that have not repaid a loan in the past so you basically get one freebie and afterwards if you didn't repay a loan in the past then you will not get a loan in the future you will still be part of the social credit system and your social credit score will still be important for all of these after-access issues but it won't be important for access to loans anymore once you've been on this blacklist which is probably something that the chinese government could go behind but it's also more more effort to take care of it then you have to think about well you can't leave them on the blacklist forever so how long do you leave them on the blacklist do they have to pay back the loan and then they get off the blacklist or do they have to pay back the loan and then stay and then like stay not not in default for a year or for five years there's a lot of so small decisions that in my opinion the chinese government hasn't really thought about up until now because they they're basically doing all these pilot studies and all of these regional governments are thinking of all these small things but they're not documenting everything that they're doing so once they they want to roll it out by 2020 by the way nationwide once they've already out there's a pretty big chance in my opinion that they have a lot of unintended consequences a lot of things that they haven't thought about and that they will then have to look at we also um yeah so i believe that some sort of system is likely to come just in terms of how much energy they've expanded into this one and for the chinese government at this point for the party it would be losing face if they did not include any such system because they've been talking about this for a while but most likely it would be a kind of decentralized data sharing system and when i ran my simulation by the way i will um make public my code i still need some um basically i used some proprietary data for my model and i still need the the permission to publish this once i publish this one i will also tweet it and we'll put it on github for everyone to play around with if you want to and one of some of these implementation details that were very important in determining model outcomes were do do we have a relative or absolute ranking so far all of the system i looked at had absolute rankings but there is a point to be made for relative rankings do we have one score where basically if you're a chinese person you get one score or do we have different sub scores in different fields do we have people reporting behavior or do we have automatic behavior recording how do you access other people's scores how much information can you get from other people's scores currently if someone is on a blacklist for example if you have their id number again you can put it into this blacklist and then they will say oh this person is on this blacklist for not following this judges order and then it says what kind of judges order it was so most likely it will be it will be something like this the idea is that this the social credit system isn't only for individuals but also for firms and for NGOs so what kind of roles will firms play in this system i haven't looked at that in detail at this point but it will be very interesting another idea that western people often talk about is do people also rank each other currently that's not part of the system in china but it might be at one point and lastly where does the aggregation happen so we've i said that a lot of it is actually data sharing in china so um what kind of data is shared do is the raw data shared person a did something or is the aggregated data shared person a got this score at this point most of the time it is actually the raw data that is shared but that also has sort of these data privacy issues of course that i've talked about okay perfect now there's 10 more minutes thank you for your attention if you have questions remarks you can ask them now you can catch me up later you can tweet to me or send me an email whatever you're interested in thank you very much if you're watching this please ask your question to ios here on twitter we'll also try to make sure to get to those let's just go ahead and start with mic one thank you very much for this beautiful talk um i was wondering how did the chinese government companies and most of all citizens themselves respond to you doing this research or let's put it differently if you would have been in the system yourself how would your research affect your social credit score um so there's there's actually two different two different responses that i've seen when i talk to the government themselves because i was there in the government scholarship and mentioned that i'm really interested in this they basically said oh well this is just a technical system you don't really need to be concerned with this it's not very important just you know it's just a technicality it's just for us to make life more efficient and better for everyone so i assume my score would actually go down from doing this research actually but when i talk to a lot of people at universities they were also very they were very interested in my research and a lot of them mentioned that they didn't even know that the system existed thank you very much before we go to a question from our signal angel a request for all the people leaving the room please do so as quietly as possible so we can continue its q and a um the signal angel please yeah the jedix wants to know um is the score actually influenced by association with people with the lowest score meaning that uh is there any peer pressure to stay away from people with bad scores um the season credit score definitely is influenced by your friend's scores the um wrong chung score so far apparently is not influenced but it is definitely in the cards and is it is planned that it will be part of this i think um the we we chat which is the main platform it's sort of like whatsapp exactly can do a lot a lot more we chat um is still not connected to the social credit score in wrong chung once they do that it will most likely also reflect your score all right let's continue with mic 3 i have a question about your model so i'm wondering um what kind of interactions are you modeling or actions like what can the agents actually do you mention moving somewhere else and um what else okay so the way i set up my model was um i set up a multi-level model so i looked at um different kinds of levels i started out with basically they can evade taxes they can get low ends and repay low ends um they can choose where to live um and they can um follow traffic rules or not follow traffic rules and um because these were sort of four big issues that were mentioned in all of the different systems so i started out with these issues and looked at um what kind of behavior do i see i used some research that um some friends of mine actually sent out surveys to people and asked them well you're now part of the system did did your behavior change and how did it change depending on your responses um depending on your score and depending on the score system that exists and i basically used that and some other research on nudging and on behavioral adaptation to look at um how likely is it that someone would change their behavior based on the score all right let's do another question from the interwebs yeah it's actually two questions in one um how does this system work for chinese people living abroad or for non-citizens that do business in china um currently the system does not work for non-citizens that do business in china because it works through the shan fan drang you only get a shan fan drang if you're a chinese citizen or you live in china for 10 or more years so everyone who is not chinese is currently excluded chinese people not living in china if they have a shan fan drang are on this system but um there's not a lot of information all right uh mic 4 well we've come a long way since the port city's world title um can you tell us anything about the dynamic in the time dimension uh how quickly can i regain credit that was lost i would be at any observations there um so in the suining system what they actually did was they had a very very strict period so if you evaded taxes your score would be down for two years and then it would rebalance in the rung chung system they did not publish this kind of period so my assumption is that it's going to be more on a case by case basis because i looked at the chinese data i looked at the chinese policy documents um and they didn't really for most of the stuff they didn't say how long it would count for the blacklists which was kind of the predecessor that we look at currently the way it works is you stay on there until whatever the black until whatever the reason for the blacklist is has been resolved so you stay on there until you send off this apology that the judge ordered you to and then usually you still needed to apply to get off so it doesn't for blacklist it does not work that you automatically get off you need to apply you need to show that you've done what they ask you to do and then you can get off this blacklist and i assume it will be a similar sort of appeals procedure for the system all right let's go to mic 2 thank you i just wanted to ask if looking up someone else's data in detail like position etc does affect your own score currently it apparently does not or at least they haven't published that it does it might in the future but most likely it's actually behavior that they want so they want you to look up other people's scores before doing business with them they want you to basically use this to decide who you're going to associate with thank you all right do we have another question from the internet maybe yes i do stand by the question is how is this actually implemented for the offline rural population in china quite easily not at all at this point the idea is by 2020 that they will actually have all of this implemented but even for the offline or let's say offline rural population in china is getting smaller and smaller even in rural villages you have about 50 60 percent of people that are online that and most of them are online via smartphone and the smartphone is connected to the shunfeng drum so it's not very complicated to do that for everyone who's online for everyone who's offline of course this is more problematic but i think the end goal is to not have people offline at all all right let's jump right back to the microphone too please okay thank you for the very good and frightening talk so far i at first i have to correct you in one point in germany we have a similar system because we have this tax id which is set from birth on and rests 30 years after a person's death um yeah so we have a lifelong id you're right i just i don't know mine so i figured no problem but at least we could establish a similar system if we if we have a government which would want it um the second question a second question from me with um you mentioned this quanchi quanchi is it a kind of a social network i didn't understand it really yes it is a kind of social network but one that is um a lot more based on hierarchies than it is in the west so you have you have people that are above you and people that are below you and the expectation is that while it's a quid for a crow people that are above you in the hierarchy will give you less than you will give to them aha okay okay all right um unfortunately we are out of time so please give another huge applause for tony
 Alright, then it's my great pleasure to introduce Toni to you. She's going to talk about the  social credit system, which kind of feels to me like a Black Mirror episode coming to life.  So, I'm slightly nervous and really curious what we're going to learn today. So please give a huge  warm round of applause and welcome Toni.  So, good morning everyone. Before I'm going to be talking, going into my talk, I'm just going to be  presenting the Chinese translation streams for everyone who just doesn't speak English.  So, we're going to be talking about the social credit system in China.  Where the social credit system that you always hear about in Western media actually doesn't  really exist. And most of my talk will actually be talking about what all we don't know,  which could fill an entire hour or even more. But I'm just going to be focusing on some of the  most interesting things for me. First of all, a little bit about me. I'm an economist, but I'm  not only concerned with money. I'm kind of looking at economics as the study of incentives,  which means that what I'm really interested in is how humans respond to different kind of incentives.  I don't believe that humans are completely rational, but I do believe that humans do try to  maximize what they think is their best interest. Now, some words about me. I studied math,  economics, and political science in a couple of different cities all around the world.  I spent overall 19 months in China. Most recently, I was there in July on a government scholarship,  which was really, really interesting because while there, I read all of these Western newspaper  articles about the Chinese social credit system. And I went to a pretty good university and I asked  them, so what do you think about this system? And most of them basically looked at me blindly  and were like, what system? I haven't even heard of this. So that was kind of an interesting  experience to me because in the West, it's like this huge all-encompassing system. And in China,  most people that aren't directly in touch with it actually don't know anything about this.  I'm broadly interested in the impact of technology and society, life, and the economy, obviously.  And in my free time, I do a lot of data science and machine learning with Python and R.  So I thought it was quite interesting to look at the social credit system also from this point of  view because you always heard that it's like this big data initiative. And then when coming down to  it, what you actually see is that they don't actually use machine learning all that much.  They have basically a rule-based catalog where if you do this, you get 50 points. If you do this,  you get 50 points. And then they actually have a lot of people that are reporting on other people's  behavior. I'm going to be talking about how exactly it looks later on. But I was very,  very surprised after reading a lot of the Western newspaper articles that were basically,  oh, this is this big dystopia or well-yain with big data working. And then you read what's actually  happening. And they have huge lists of if you, J-Wog, you get 10 points distracted from you,  this kind of thing. If you want to get in touch with me, you can use Twitter, but you can also  use different emails, either my professional email or my personal email address that you can both see  there. If you have any thoughts on that or are interested in this a little more, I can give you  more resources as well because obviously today's talk will only be scratching on the surface.  So perceptions of the social credit system. One of the interesting things that I've talked about  before was how in the West and in China, the perception is completely different. So in the  West, which is from FinancialTimes.com, you see this huge overwhelming guy and he basically puts  every Chinese person under a microscope. They're all kind of hunched over and everyone has this  score attached to them and they seem pretty sad and like very, very Orwellian concept. Whereas  in China, this is actually from a Chinese state media. And what it says is, well, we can all  live in harmony with this new system and all trust each other. And interestingly, Chinese people  actually believe that to some degree. They believe that technology will fix all this current problems  in the in society, especially because in China currently trust is a rare commodity.  And this new system will lead to more efficiency and trust and a better life. And I have a really,  really interesting quote from a Western scholar that really summarizes the Western perspective.  What China is doing here is selectively reading its population to select against that, the trade  of critical independent thinking. This may not be the purpose. Indeed, I doubt it's the primary  purpose, but it's nevertheless the effect of giving only obedient people the social ability to have  children, not to mention successful children. This basically plays with the idea that if you  have a low score currently in the cities that are already testing this system, what happens is  your children can't attend good schools. What happens is you cannot take trains, you cannot  take planes, you cannot book good hotels. Your life is just very, very inconvenient. And this  is by design. This is kind of the plan. The Chinese government, they say it's a little  different. The idea is about changing people's conduct by ensuring they are closely associated  with it. One of the main things about this system is there isn't very much new data being generated  for the system. Instead, what's happening is all the existing data that is already collected about you  is basically combined into one big database for each and every person by your ID number. So in  China, once you're born, you get an ID number, which is similar to a social security number in the  US. We don't really have a similar concept in Germany. And it used to be that your ID number was  only necessary for public, like for government stuff, but now you need your ID number for getting a  bank account. You need your ID number for buying a cell phone. Even if it's a prepaid cell phone,  you still need your ID number. So all your online activity that happens with your cell phone is  associated with your ID number, which means you can't really do anything anonymously because it's  all going back to your ID number. There's a couple of predecessors, some of them going actually back  to the 1990s, that are supposed to be integrated into the new system. One of them or like two of  them are blacklists. One of them is a court blacklist. So in China, courts work a little bit  differently. They tend to like giving you fines as they do in other countries, but they also like  giving you apologies to do. So one of the things, if you do something, for example, you're a company,  your food safety wasn't up to par, you have to pay a fine. But in addition to this fine, you also  have to write a public apology letter in the newspaper. How you're very sorry that this happened  and it won't happen again. And it was a moral failing on your part and it won't happen again.  And if you don't do that, you go on this blacklist. Similarly, if you take out a line of credit and  don't pay it back within three months or don't do any payments for three months, you go on this  debtor's blacklist. If you're on this blacklist, which again is associated with your Shenfeng  drone, so your ID number, what happens is you cannot take trains, you cannot take planes. Your  life basically becomes very, very inconvenient. Your children can't go to good public schools,  your children can't go to private schools, your children can't go to universities. All of these  issues are suddenly coming up. There is also a company database that's called Credit China,  which is basically similar to the public debtor's blacklist, but it's basically a credit system,  a credit score for companies. And then there's the credit reference center of the People's Bank  of China, which is a credit score. It was supposed to be like Shufa or like the US FICO for  individuals. But one of the big problems in China is that there are a lot of people that  aren't part of the formal economy. A lot of people are migrant workers. They get their money in cash.  They do not have bank accounts. They do not have anything, they do not have rent or utilities or  anything like this because they live in the country, so they own their own home, which they built  themselves so they didn't even finance it. And their home isn't officially theirs because in China  you can't actually own property. Instead, the government leases it to you. So there were a lot  of people that were not covered in this system. And I think the last data that I had was that  less than 10% of Chinese adult citizens were actually in the system and had any sort of exposure  to banks, which is very, very little. And that meant that people couldn't get credit because  banks would only give credit to people that were in the system or people where they had  some sort of handling on whether they would be paid back. Now, the implementation details of the  new system are very, very scarce. But the basic idea is that Chinese citizens are divided into  trustworthy individuals and what the Chinese call trust breakers. Sometimes you have five  different groups, sometimes you have two different groups, but in general there's sort of this cut  off above this line, it's good and beyond this line, it's bad. This is one graphic from the  Wall Street Journal that just shows some of the inputs that go into the system. And one of the  things that we see is that the inputs are crazy, crazy varied. So it is, do you pay income taxes?  Do you pay your utility bills on time? Do you respect your parents? However they  measure that. Do you have a criminal record? Do you pay for public transportation or have you been  caught not paying? What about your friends? Do you retweet or use WeChat to  distribute sort of information against the party which they call reliability? In actuality it's  not about whether it's factual, it's about whether it's against the party or not. Where do you buy  and what do you buy? Apparently if you buy diapers it's better than if you buy video games  for your score because you know if you buy video games obviously you're not very responsible  and if you buy diapers you have a kid you're sort of conforming to the societal idea. And then  your score is supposed to go into all these different categories. You're supposed to have  better access to social services if your score is good. You're supposed to have better access to  internet services. So in theory the idea is that at one point if your score is too bad you're not  allowed to use WeChat anymore. You're not allowed to use Alibaba anymore. You can't become a government  worker. You cannot take planes and high-speed trains. You cannot get a passport and your insurance  premiums will go up. So it's supposed to be this really really big overwhelming system but  in actuality what they say their stated goals are is it's a shorthand for a broad range of  efforts to improve market security and public safety by increasing integrity and mutual trust  in society. So one idea is to allocate resources more efficiently. Resource allocation in China is  a pretty big problem because people grow up with there's 1.3 billion people so there's it's always  going to be scars and a lot of stuff is people grow up with this idea that it's just very very  scars and current distribution strategies which are mostly financially based but also often  one-T based don't really seem fair. For example public transport in China is highly subsidized  which means that the price does not reflect whether does not reflect true scarcity. So currently the way  it works is in theory it's first come first serve and practice there's people that are buying up all  the tickets for for example the high-speed train from Shanghai to Beijing and then selling it at a  profit or selling it to certain companies that have good ties to the government. That seems very  unfair so the new system is supposed to distribute them more fairly and more efficiently. The other  thing is restoring trust in people. Perceived interpersonal trust and trust in institutions  is extremely low in China. If you're from Germany you might have heard that there's Chinese gangs  basically buying up German milk powder and selling it in China. This is actually happening because  in 2008 there was a big scandal with laced milk powder and ever since then anyone who can afford  it does not use Chinese milk powder because they don't trust the government or the regulations the  firms enough to buy Chinese milk powder. So they're actually importing this and the the big irony is  sometimes this milk powder is produced in China exported to Germany and then exported back to China.  The social credit system is then supposed to identify those that deserve the trust and the third  point is sort of a reeducation of people. The idea is they want to make people in the image that  the Communist Party thinks people should be and one additional way to the punishments and rewards  this could work is the feeling of being surveyed because you can't do anything anonymously. You  will automatically adapt your behavior because you know someone is watching you all the time  and this is how a lot of the Chinese firewall actually works because most people I know that  are sort of more more educated they know ways to circumvent the Chinese firewall but they also know  that they're always being watched so they don't do that because you know they're being watched so  they self they censor ship they censor themselves. As I've said before allocation of scarce resources  so far is mainly through financial or guanxi channels. Guanxi is basically an all-permeeting  network of relationships with a clear status hierarchy. So if I attend a school everyone who  also attended this school will be sort of in my guanxi network and there's this idea that we will  have a system where we are all in group and in group we trust each other and we do favors for  each other and everyone who's outside of my immediate group doesn't I don't trust and I don't do favors  for and in some ways the guanxi system right now is a substitute for formal institutions in China.  For example if you want a passport right now you can of course apply for a passport through  regular channels which might take months and months or you can apply for a passport through  knowing someone and knowing someone which might take only two days whereas in Germany you have  these very regular formal institutions in China they still use guanxi but increasingly especially  young people find that guanxi are very unfair because a lot of these are who are where you went  to school which is determined by where you're born who your parents are and all these things.  Another thing that's important to understand because the system works through public shaming  and in a lot of western society we can't really imagine that like I wouldn't really care if my name  was in a newspaper of someone who jaywalked for example it would be oh well that's okay but in  China this is actually a very very serious thing so saving face is very very important in China  and when I went to school there I actually we had this dormitory and it was a all foreigners dormitory  where the staff that were responsible for the dormitory felt that foreigners were not behaving  in the way they should so their idea was to put the names the pictures and the offenses of the  foreigners in the elevator to shame them publicly so for example if you brought a person of the  opposite sex to your room they would put your name your offense and your room number in the elevator  and of course this didn't work because for a lot of western people it was basically like oh well  I'm going to try to be there as often as possible because this is like a batch of honor for me and  the Chinese people they figured well this is really really shame and I'm losing my face  she brought alcohol so this didn't really work at all but this is kind of the mindset  that is behind a lot of these initiatives as I've said there's a lot of problems with we don't  really know what's going to happen and one of the ways that we can see what might happen is actually  to look at pilot systems China has or like the ever since the communist party took hold that  the Chinese government has tried a lot of policy experimentation so whenever they try a new policy  they don't roll it out all over but they choose different pilot cities or pilot districts and  then they choose oh well this is the district where I'm going to be trying this system and  I'm going to be trying another system in another district or city and this is also what they did  for the what they're doing for the social credit system now I have three systems that I looked at  intensively for this presentation overall there's about 70 that I know of the swimming system  swimming is a city in in China the wrong trunk system another city in China and season credit  season credit is a commercial system from Alibaba I assume everyone knows Alibaba they're basically  the Chinese Amazon except they're bigger and have more users and make more money actually and they  have their own little system one of the problems with this kind of system that I found when I tried  modeling it was that it's a very very complex system and small changes in input actually changed  the output significantly so when they try usually when they try this pilot system they basically  have a couple of pilots then they choose the pilot that is best and they roll it out all over but for  this kind of thing where you have a lot of complex issues it might not be the best way to do that  the swimming system is actually considered the predecessor of all current systems it had a focus  on punishment and it was quite interesting at the beginning of the trial period they published a  catalog of scores and consequences here is an example this is basically taken from this catalog  so if you took out bank loans and didn't repay them you've got deducted 50 points everyone started  with a thousand points for this system if you didn't pay pay back your credit cards you also got  deducted 50 points if you evaded taxes also 50 points if you sold fake goods 35 points were  deducted and actually the system was abolished I think in 2015 2016 because all the the Chinese  state media and also a lot of internet citizens talked about how it's an Awellian system and how  it's not a good system because it's all very centralized and everything that you do is basically  recorded centrally but Krimers writes nonetheless the swimming system already contained the embryonic  forms of several elements of subsequent social credit initiatives the notion of dispersional  disincentives against rule breaking public naming and shaming of wrongdoers and most importantly  the expansion of credit mechanism outside of the market economic context also encompassing  compliance with administrative regulation and urban management rules so one of the things that is  difficult for especially German speakers is that credit in Chinese Xinyou means credit as in loan  but also means credit as in trust so the social credit system is one way of trying to conflict  those two the economic credit and the trust credit into one big system but the swimming system  basically failed so they adapted the system and are now practicing a new kind of system the wrong  term system whenever you read a newspaper article on the social credit system in the west most people  went to wrong term because they just received a couple of awards from the Chinese government for  being so advanced at this social credit thing but it's it's very difficult to call this one system  because there's actually many many intertwined system there is one city level system where city  level offenses are recorded for example tax evasion and there's a couple of there's a couple of rules  if you evade taxes your score goes down 50 but then if you live in one neighborhood your score  might go up for volunteering with the elderly if you live in another neighborhood your score might go  up for for example planting some trees in your garden or backyard so depending on your name  of your score might be different if you work for a if you work for a taxi cab company for example  they also have their own little score system and your score might go up if you get good reviews  from your drive from your passengers your score might go down if you don't follow traffic rules  these kinds of things there are and there are designated scorekeepers at each level so each  district chooses a couple of people who are responsible for passing on the information to the next  higher level about who did what there is supposed to supposed to be an official appeals procedure  so whenever your score changes you're supposed to be notified but apparently that's not happening  at this point for most people um again it's a system of data sharing and one thing that they  haven't really disclosed yet is what kind of data is shared are they only sharing the points  so if i'm in a district and i plant some trees does the central system get the information  person a planted some trees or does the central system get the information person a got five points  we don't know at this point and it would mean something very different for how the system  could be used but still the end result at this point is that there's one score so you have one  central score and it's kind of there's all these different smaller systems that go into this score  but at the end everyone has one central score and currently about 85 percent of people are between  950 and 1050 so you start up you start off with 1000 and those are basically the normal people  and then anyone above 1050 is considered a trust a trustworthy person and anyone below 1050 is  considered a trust breaker and as i've said before with the naming and shaming and all these things  what you can actually see here is a billboard with the best trustworthy families in wrong  throne so these are the families that have the highest scores for example season credit is a  little different it's the only system that actually uses machine learning and artificial  intelligence to determine the outputs in wrong chung for example they have artificial intelligence  they have computer vision for the most part and the computer vision cameras they decide  they try to recognize you when you jaywalk and then when they recognize you when jaywalking you get  a small sms well we just saw you jaywalking your score is now dropping but how far how the score  develops depending on your jaywalking isn't really determined by machine learning or artificial  intelligence instead it's determined by rules you know well one time jaywalking deducts five  points and this is stated somewhere um season credit doesn't work like that instead it uses  a secret algorithm and the way i talked to some people that work for season credit or for alibaba  and the way they described it was they basically clustered their clustered people based on behavior  then gave scores to these clusters and then afterwards um did basically reverse engineered  their own score using machine learning so that whenever something new happens you can move to  a different cluster um this season credit was actually refused accreditation as a as a credit  score um in 2017 so banks are not allowed to use the season credit score for your um to use the  season credit score to determine whether they give you loans or not because season credit is  quite ingenious obviously alibaba wants to keep you within their platform so if you buy using alibaba  and using alipay your score goes up if you buy using wechat pay which is a competing platform  your score goes down and this uses many of the same rewards mechanism of the official government  systems and this is just an illustration of what kind of scores you can have apparently  your scores can go between 350 and 850 and in chinese there's basically five different  levels so 385 is um trust breaker or missing trust and then 731 is trust break trust is exceedingly high  so um one way i tried to approach this issue was through agent-based modeling social credit system  is individual level but what we are really interested in or what i'm really interested in is actually  societal level consequences so if everyone gets this score what does that mean for society  um agent-based modeling works quite well for that because it allows us to imbue agents with  some sort of rationality but with a bounded rationality what does bounded rationality mean  usually in economics people assume agents are completely rational so they are profit maximizers  they have all the information but in reality agents don't have all the informations they have a lot of  issues with um issues with keeping stuff in their mind so a lot of the time they won't choose the  best thing in the world but they choose the best thing that they see um and bounded rationality  allows us to account for this thing it allows us to account for um heuristics and these things  and what i did is i took the propensity for specific behavior from current state of the  art research mostly from behavioral economics for example um i looked at tax evasion and i looked at  who is likely to evade taxes in a in a system and then obviously there was some stochastic  some chance element but the distribution that i chose is um related to the current research  and i also checked that um my model has similar results to the wrong-tongue model which i modeled  at at the beginning um so on average 87 percent um of my users have a score of within 10 percent of  the original score which is also the data that um wrong-tongue city actually publishes  now um for the most part i compared four different i compared design choices in two axes one of them  was a centralized system versus a multi-level system and a rule-based system versus versus a  machine learning system the centralized system is basically you have a central you have um  all the information is kept centrally and um everyone in china or wherever in wrong-tongue  has the exact same scoring opportunities um now if you have a centralized system the clear  expectations were um were pretty good but at the same time the acceptance from the population was  really really low um which they found during the swimming experiment and there's also the problem  of a single point of failure who decides the central catalog and um depending on who sort of has the  power it kind of just um reproduces power structures so because you have this central catalog  the people that are in power centrally they are basically deciding some sort of score  mechanism that works for them very well so that they and their family will have high scores  a multi-level system um has the advantage that local adaptation kind of works and there's sort  of many points of failure but um in my model when I allowed locals to basically set their own rules  what happened was that they competed so it started out being this district of wrong-tongue for example  and this district of wrong-tongue they compete for the best people that they want to attract  and suddenly you have this kind of race to the bottom where um people want to move where they  wouldn't be prosecuted so they move to places where there's less cameras for example at the same  time there's many points of failures especially the way it's currently set up with um people  reporting data to the next to the next high level and a lot of the time what we have actually seen  in wrong-tongue was that they reported data on people they didn't like more than data on people  they did like or their families got better scores than people they didn't know so it also kind of um  reproduced these biases the um rule-based system has the advantage that um people were more prone  to adapt their behavior because they actually knew what they needed to do in order to adapt their  behavior but the score didn't really correlate with the important characteristics that they actually  cared about and um as opposed to if in this machine learning system you know how in Germany  we don't really know the shufa algorithm and I for example don't exactly know what I could do  in order to improve my shufa score and this is a similar system in China with the sesam credit  score a lot of people don't really they they say well I really want to adapt my behavior to the  score to improve my score but when I tried doing that my behavior my score actually got worse  and you can have different biases that I'm going to be talking about in a little bit  um there's also this big problem of incentive mismatch so um the decentralized rules-based  systems like rongchong which is the system that I analyze the most why because I believe this is  the system that we're moving towards right now because rongchong won a lot of awards so the  Chinese government the way they usually work is they try pilots then they choose the best couple  of systems they give them awards and then they they roll out the system nationwide so I assume  that the system that's going to be the system in the end will be similar to the rongchong system  now one problem that I actually saw in my um simulation was that you could have this possible  race to the bottom um there's also this conflict of interest in those that set the rules because  a lot of the time the way it works is you have come you have your company in your company you in  in combination with your um party leaders actually decide on the rules for your um for the score  system but the scores of all your employees actually determines your company's score if you  employ a lot of people with high scores you get a better score so you have this incentive to give  out high scores and to make sure that everyone gets high scores but at the same time the government  has an incentive for scores to be comparable um so there's a lot of int incentives mismatch  the government also has the incentive to keep false negatives down but they actually the way  the Chinese system currently works is um they emphasize catching trust breakers more than rewarding  trust follow or like trustworthy people so um false positives for them are less important  but false positives erode the trust in the system and they lead to a lot less behavioral adaptation  I was actually able to show this using some um nudging research where that showed that as soon  as you introduce a couple an error probability and you can be caught for something that you didn't do  your your probability of um changing your behavior based on the score are actually is actually lower  and in Rongchang one of the perverse things that they're doing is you can um you can donate money  to the party or to party to like party affiliated um social services and this will give you points  um which is kind of an indulgence system which is quite interesting especially because a lot of  these donation systems work that way work in a way that you can donate 50 000 renminbi and you  get 50 points and then you donate another 50 000 renminbi and you get another 50 points so you can  basically donate a lot of money and then behave however you want and still get a good score  and um the trust in other people can actually go down even more in the system because suddenly  you only trust them because of their scores and um the current system is set up so that you can  actually look up scores of everyone that you want to work with and if they don't have a score high  enough then suddenly you don't want to work with them um the trust in the legal system can also  decrease actually um why because trust in the legal system in china is already low and a lot of the  things like jaywalking they're already illegal in china as they are here but no one cares and  suddenly you have this parallel system that um punishes you for whatever but why do you why  don't you just try to fix the legal system which would be my approach you can suddenly illegal  activity could happen more offline and this is one of those things that um is quite interesting  in countries that we've seen that have moved towards mobile payments and away from cash  you see less robberies but so you don't see less um you don't actually see less crime instead you see  more new types of crime so you see more credit card fraud you see more um phone robberies these  kinds of things and this is also where um things could move for um that in the chinese case  one major problem is also that this new system i've talked a little bit about this one but it  can introduce a lot of new bias and reproduce the bias even more so um for example um china is a  country of 55 minorities the hun are a big majority they have about 94 percent of the population so  any computer vision task we've shown that they are really really bad at discriminating or at  discriminating between individuals in um smaller ethnic groups in the u.s most computer vision  tasks perform worse for african americans they perform worse for women because all of the training  sets are male and white and maybe asian in china all of these tasks are actually um performing  worse for ethnic minorities for the yuigurs for example and one way that they could try to abuse  the system is to basically just what they also what they're also doing already in qinjiang is to  basically just identify oh this is a person of the minority well i'm just gonna go i'm just gonna  go and check him or her more more thoroughly this is actually what happens in qinjiang if you're in  qinjiang and you look like a turkish person or like from turkish people um you can you are a  lot more likely to be questions you're a lot more likely to be stopped and they ask you or  require you to download spyware on your phone um and this is currently what happens and this new  kind of system can actually help you with that um i've said that it can reproduce these kind of power  structures and now obviously we all know newtville technology doesn't really exist but in the chinese  case in the social credit case they don't even pretend they always say well this is newtville  technology and it's all a lot better but actually it's the people currently in power they decide on  what gives what gives you point and what detect what deducts points for you  at the problem currently the the entire system is set up in a way that it's all goes together with  your shan fendrong with your id card what if you don't have an id card that's foreigners for one  but it's also people in china that were born during the one child policy and were not registered  there's quite a lot of them actually they're not registered anywhere and suddenly they can't do anything  because they don't have a score they can't get a phone they can't do anything really and um part of  the part of the push with this um social credit system is to go away from cash actually so if you  you need to use your phone to pay but for your phone you need your shan fendrong if you don't have a  shan fendrong well tough luck for you and um currently the system in wrong trung is set up in  a way that you can check other people's scores and you can also see what they lose points for  so you can actually sort of choose to discriminate against people that are gay for example because  they might have lost points for going to a gay bar which you can lose points for  another big issue currently is data privacy and security um personal data is grossly undervalued  in china if you ask a chinese person what do you think how much is your data worth they're  going to be saying what data i don't have data um and currently the way it works is if you have  someone's id number which is quite easy to find out um you can actually buy access to a lot of  personal information for a small fee so you pay about a hundred euros and you get um all hotel  bookings of the last year you get information of who booked these hotels with them you get information  of where they stayed you get trained bookings you get access to all of the official databases for  this one person and for another 700 rendman b you can actually get live location data so you  can get the data of where this person is right now or where he's on her phone is right now but if  you've ever been to china you know that where the phone is usually the people aren't far um  soup china actually did an experiment where a couple of journalists tried buying that because  it's actually these kind of services are offered on we chant pretty publicly and you can just buy  them quite easily um so one additional thing that i looked at is um because one of the things that  is quite interesting is you have this idea of credit as twofold credit is um trust credit but  credit is also low in credit and what if credit institutions actually use this unified credit  score to determine credit distribution the idea is that it's supposed to lead to reduced information  asymmetry obviously so fewer defaults and overall more credit creation new people are supposed to  get access to credit and they're supposed to be less shadow banking but what actually happens  i'm not going to be talking about how i set up the model but just about my results if you have  this kind of score that includes credit information but also includes morally good or measures of being  morally good what you have is in the beginning about 30 more agents get access to credit and  especially people that previously have not gotten credit access suddenly have credit access but the  problem is that this social credit score that correlates all of these different issues um  it correlates only very very weakly with repayment ability or repayment wishes and thus suddenly  you have all of these non-performing loans um you have and what we see is sort of like we have  non-performing loans banks give out less loans because they have so many non-performing loans  and um then the non-performing loans are written off and suddenly banks give out more loans but you  have this oscillating financial system where you have where you give out a lot of loans a lot of them  are non-performing then you give out a lot of loans again and this is very very vulnerable to crisis  if you have a real economic crisis during the time where non-performing loans are high then a lot of  banks will actually default which is very very dangerous for a financial system as nationed as  the Chinese one now what are some possible corrections you could create a score that um  basically is the same as a shoofa score so that looks only at credit decisions but suddenly  you lose a lot of incentives for the social credit score if the social credit score doesn't matter  for credit distribution anymore another ish thing and this is i think the more likely one is that  you have a blacklist for people that have not repaid a loan in the past so you basically get one  freebie and afterwards if you didn't repay a loan in the past then you will not get a loan in the  future you will still be part of the social credit system and your social credit score will still be  important for all of these after-access issues but it won't be important for access to loans anymore  once you've been on this blacklist which is probably something that the chinese government  could go behind but it's also more more effort to take care of it then you have to think about  well you can't leave them on the blacklist forever so how long do you leave them on the blacklist  do they have to pay back the loan and then they get off the blacklist or do they have to pay back  the loan and then stay and then like stay not not in default for a year or for five years  there's a lot of so small decisions that in my opinion the chinese government hasn't really  thought about up until now because they they're basically doing all these pilot studies and all  of these regional governments are thinking of all these small things but they're not documenting  everything that they're doing so once they they want to roll it out by 2020 by the way  nationwide once they've already out there's a pretty big chance in my opinion that they have a lot  of unintended consequences a lot of things that they haven't thought about and that they will then  have to look at we also um yeah so i believe that some sort of system is likely to come  just in terms of how much energy they've expanded into this one and for the chinese government at  this point for the party it would be losing face if they did not include any such system  because they've been talking about this for a while but most likely it would be a kind of  decentralized data sharing system and when i ran my simulation by the way i will um make public my  code i still need some um basically i used some proprietary data for my model and i still need the  the permission to publish this once i publish this one i will also tweet  it and we'll put it on github for everyone to play around with if you want to and one of  some of these implementation details that were very important in determining model outcomes were  do do we have a relative or absolute ranking so far all of the system i looked at had absolute  rankings but there is a point to be made for relative rankings do we have one score where  basically if you're a chinese person you get one score or do we have different sub scores in different  fields do we have people reporting behavior or do we have automatic behavior recording  how do you access other people's scores how much information can you get from other people's scores  currently if someone is on a blacklist for example if you have their id number again you can put it  into this blacklist and then they will say oh this person is on this blacklist for not following  this judges order and then it says what kind of judges order it was so most likely it will be  it will be something like this the idea is that this the social credit system isn't only for  individuals but also for firms and for NGOs so what kind of roles will firms play in this system  i haven't looked at that in detail at this point but it will be very interesting  another idea that western people often talk about is do people also rank each other currently  that's not part of the system in china but it might be at one point and lastly where does the  aggregation happen so we've i said that a lot of it is actually data sharing in china so um  what kind of data is shared do is the raw data shared person a did something or is the  aggregated data shared person a got this score at this point most of the time it is actually  the raw data that is shared but that also has sort of these data privacy issues of course that i've  talked about okay perfect now there's 10 more minutes thank you for your attention if you  have questions remarks you can ask them now you can catch me up later you can tweet to me or send  me an email whatever you're interested in thank you very much  if you're watching this please ask your question to ios here on twitter we'll also try to make  sure to get to those let's just go ahead and start with mic one thank you very much for this  beautiful talk um i was wondering how did the chinese government companies and most of all  citizens themselves respond to you doing this research or let's put it differently if you  would have been in the system yourself how would your research affect your social credit score  um so there's there's actually two different two different responses that i've seen  when i talk to the government themselves because i was there in the government scholarship and  mentioned that i'm really interested in this they basically said oh well this is just a technical  system you don't really need to be concerned with this it's not very important just you know it's  just a technicality it's just for us to make life more efficient and better for everyone  so i assume my score would actually go down from doing this research actually but when i talk to a  lot of people at universities they were also very they were very interested in my research and a lot  of them mentioned that they didn't even know that the system existed thank you very much  before we go to a question from our signal angel a request for all the people leaving the room  please do so as quietly as possible so we can continue its q and a um the signal angel please  yeah the jedix wants to know um is the score actually influenced by association with people  with the lowest score meaning that uh is there any peer pressure to stay away from people with  bad scores um the season credit score definitely is influenced by your friend's scores the um  wrong chung score so far apparently is not influenced but it is definitely in the cards and  is it is planned that it will be part of this i think um the we we chat which is the main platform  it's sort of like whatsapp exactly can do a lot a lot more we chat um is still not connected to the  social credit score in wrong chung once they do that it will most likely also reflect your score  all right let's continue with mic 3 i have a question about your model so i'm wondering um  what kind of interactions are you modeling or actions like what can the agents actually do  you mention moving somewhere else and um what else okay so the way i set up my model was um i set up  a multi-level model so i looked at um different kinds of levels i started out with basically they  can evade taxes they can get low ends and repay low ends um they can choose where to live um and  they can um follow traffic rules or not follow traffic rules and um because these were sort of  four big issues that were mentioned in all of the different systems so i started out with these issues  and looked at um what kind of behavior do i see i used some research that um some friends of mine  actually sent out surveys to people and asked them well you're now part of the system did  did your behavior change and how did it change depending on your responses um depending on your  score and depending on the score system that exists and i basically used that and some other  research on nudging and on behavioral adaptation to look at um how likely is it that someone would  change their behavior based on the score all right let's do another question from the interwebs  yeah it's actually two questions in one um how does this system work for chinese people living  abroad or for non-citizens that do business in china um currently the system does not work for  non-citizens that do business in china because it works through the shan fan drang you only get a  shan fan drang if you're a chinese citizen or you live in china for 10 or more years so everyone who  is not chinese is currently excluded chinese people not living in china if they have a shan fan drang  are on this system but um there's not a lot of information  all right uh mic 4 well we've come a long way since the port city's world title um can you tell us  anything about the dynamic in the time dimension uh how quickly can i regain credit that was lost  i would be at any observations there um so in the suining system what they actually did was  they had a very very strict period so if you evaded taxes your score would be down for two years  and then it would rebalance in the rung chung system they did not publish this kind of period  so my assumption is that it's going to be more on a case by case basis because i looked at the  chinese data i looked at the chinese policy documents um and they didn't really for most of the stuff  they didn't say how long it would count for the blacklists which was kind of the predecessor  that we look at currently the way it works is you stay on there until whatever the black  until whatever the reason for the blacklist is has been resolved so you stay on there until you  send off this apology that the judge ordered you to and then usually you still needed to apply to  get off so it doesn't for blacklist it does not work that you automatically get off you need to  apply you need to show that you've done what they ask you to do and then you can get off this blacklist  and i assume it will be a similar sort of appeals procedure for the system all right let's go to  mic 2 thank you i just wanted to ask if looking up someone else's data in detail like position etc  does affect your own score currently it apparently does not or at least they haven't published that  it does it might in the future but most likely it's actually behavior that they want so they want  you to look up other people's scores before doing business with them they want you to basically use  this to decide who you're going to associate with thank you all right do we have another question from  the internet maybe yes i do stand by the question is how is this actually implemented for the offline  rural population in china quite easily not at all at this point the idea is by 2020 that they will  actually have all of this implemented but even for the offline or let's say offline rural population  in china is getting smaller and smaller even in rural villages you have about 50 60 percent of  people that are online that and most of them are online via smartphone and the smartphone is connected  to the shunfeng drum so it's not very complicated to do that for everyone who's online for everyone  who's offline of course this is more problematic but i think the end goal is to not have people offline  at all all right let's jump right back to the microphone too please okay thank you for the very  good and frightening talk so far i at first i have to correct you in one point in germany we  have a similar system because we have this tax id which is set from birth on and rests 30 years  after a person's death um yeah so we have a lifelong id you're right i just i don't know mine so i  figured no problem but at least we could establish a similar system if we if we have a government  which would want it um the second question a second question from me with um you mentioned this  quanchi quanchi is it a kind of a social network i didn't understand it really yes it is a kind of  social network but one that is um a lot more based on hierarchies than it is in the west  so you have you have people that are above you and people that are below you and the expectation is  that while it's a quid for a crow people that are above you in the hierarchy will give you less  than you will give to them aha okay okay all right um unfortunately we are out of time so please  give another huge applause for tony
35c3-9904-the_social_credit_system_0_59.png 35c3-9904-the_social_credit_system_60_1175.png 35c3-9904-the_social_credit_system_1176_1955.png 35c3-9904-the_social_credit_system_1956_2351.png 35c3-9904-the_social_credit_system_2352_2423.png 35c3-9904-the_social_credit_system_2424_4931.png 35c3-9904-the_social_credit_system_4932_8735.png 35c3-9904-the_social_credit_system_8736_9419.png 35c3-9904-the_social_credit_system_9420_9743.png 35c3-9904-the_social_credit_system_9744_9839.png 35c3-9904-the_social_credit_system_9840_14423.png 35c3-9904-the_social_credit_system_14424_16799.png 35c3-9904-the_social_credit_system_16800_24347.png 35c3-9904-the_social_credit_system_24348_27071.png 35c3-9904-the_social_credit_system_27072_27779.png 35c3-9904-the_social_credit_system_27780_30299.png 35c3-9904-the_social_credit_system_30300_36503.png 35c3-9904-the_social_credit_system_36504_36575.png 35c3-9904-the_social_credit_system_36576_36971.png 35c3-9904-the_social_credit_system_36972_42755.png 35c3-9904-the_social_credit_system_42756_46319.png 35c3-9904-the_social_credit_system_46320_49943.png 35c3-9904-the_social_credit_system_49944_50315.png 35c3-9904-the_social_credit_system_50316_50411.png 35c3-9904-the_social_credit_system_50412_50843.png 35c3-9904-the_social_credit_system_50844_51755.png 35c3-9904-the_social_credit_system_51756_54623.png 35c3-9904-the_social_credit_system_54624_54731.png 35c3-9904-the_social_credit_system_54732_54995.png 35c3-9904-the_social_credit_system_54996_55499.png 35c3-9904-the_social_credit_system_55500_56027.png 35c3-9904-the_social_credit_system_56028_58883.png 35c3-9904-the_social_credit_system_58884_58967.png 35c3-9904-the_social_credit_system_58968_61931.png 35c3-9904-the_social_credit_system_61932_63911.png 35c3-9904-the_social_credit_system_63912_70835.png 35c3-9904-the_social_credit_system_70836_70883.png 35c3-9904-the_social_credit_system_70884_70955.png 35c3-9904-the_social_credit_system_70956_72683.png 35c3-9904-the_social_credit_system_72684_77399.png 35c3-9904-the_social_credit_system_77400_77603.png 35c3-9904-the_social_credit_system_77604_78407.png 35c3-9904-the_social_credit_system_78408_79235.png 35c3-9904-the_social_credit_system_79236_81851.png 35c3-9904-the_social_credit_system_81852_85163.png 35c3-9904-the_social_credit_system_85164_85991.png 35c3-9904-the_social_credit_system_85992_88295.png 35c3-9904-the_social_credit_system_88296_90227.png 35c3-9904-the_social_credit_system_90228_90731.png 35c3-9904-the_social_credit_system_90732_91583.png 35c3-9904-the_social_credit_system_91584_91751.png 35c3-9904-the_social_credit_system_91752_91904.png
Alright, then it's my great pleasure to introduce Toni to you. She's going to talk about the social credit system, which kind of feels to me like a Black Mirror episode coming to life. So, I'm slightly nervous and really curious what we're going to learn today. So please give a huge warm round of applause and welcome Toni. So, good morning everyone. Before I'm going to be talking, going into my talk, I'm just going to be presenting the Chinese translation streams for everyone who just doesn't speak English. So, we're going to be talking about the social credit system in China. Where the social credit system that you always hear about in Western media actually doesn't really exist. And most of my talk will actually be talking about what all we don't know, which could fill an entire hour or even more. But I'm just going to be focusing on some of the most interesting things for me. First of all, a little bit about me. I'm an economist, but I'm not only concerned with money. I'm kind of looking at economics as the study of incentives, which means that what I'm really interested in is how humans respond to different kind of incentives. I don't believe that humans are completely rational, but I do believe that humans do try to maximize what they think is their best interest. Now, some words about me. I studied math, economics, and political science in a couple of different cities all around the world. I spent overall 19 months in China. Most recently, I was there in July on a government scholarship, which was really, really interesting because while there, I read all of these Western newspaper articles about the Chinese social credit system. And I went to a pretty good university and I asked them, so what do you think about this system? And most of them basically looked at me blindly and were like, what system? I haven't even heard of this. So that was kind of an interesting experience to me because in the West, it's like this huge all-encompassing system. And in China, most people that aren't directly in touch with it actually don't know anything about this. I'm broadly interested in the impact of technology and society, life, and the economy, obviously. And in my free time, I do a lot of data science and machine learning with Python and R. So I thought it was quite interesting to look at the social credit system also from this point of view because you always heard that it's like this big data initiative. And then when coming down to it, what you actually see is that they don't actually use machine learning all that much. They have basically a rule-based catalog where if you do this, you get 50 points. If you do this, you get 50 points. And then they actually have a lot of people that are reporting on other people's behavior. I'm going to be talking about how exactly it looks later on. But I was very, very surprised after reading a lot of the Western newspaper articles that were basically, oh, this is this big dystopia or well-yain with big data working. And then you read what's actually happening. And they have huge lists of if you, J-Wog, you get 10 points distracted from you, this kind of thing. If you want to get in touch with me, you can use Twitter, but you can also use different emails, either my professional email or my personal email address that you can both see there. If you have any thoughts on that or are interested in this a little more, I can give you more resources as well because obviously today's talk will only be scratching on the surface. So perceptions of the social credit system. One of the interesting things that I've talked about before was how in the West and in China, the perception is completely different. So in the West, which is from FinancialTimes.com, you see this huge overwhelming guy and he basically puts every Chinese person under a microscope. They're all kind of hunched over and everyone has this score attached to them and they seem pretty sad and like very, very Orwellian concept. Whereas in China, this is actually from a Chinese state media. And what it says is, well, we can all live in harmony with this new system and all trust each other. And interestingly, Chinese people actually believe that to some degree. They believe that technology will fix all this current problems in the in society, especially because in China currently trust is a rare commodity. And this new system will lead to more efficiency and trust and a better life. And I have a really, really interesting quote from a Western scholar that really summarizes the Western perspective. What China is doing here is selectively reading its population to select against that, the trade of critical independent thinking. This may not be the purpose. Indeed, I doubt it's the primary purpose, but it's nevertheless the effect of giving only obedient people the social ability to have children, not to mention successful children. This basically plays with the idea that if you have a low score currently in the cities that are already testing this system, what happens is your children can't attend good schools. What happens is you cannot take trains, you cannot take planes, you cannot book good hotels. Your life is just very, very inconvenient. And this is by design. This is kind of the plan. The Chinese government, they say it's a little different. The idea is about changing people's conduct by ensuring they are closely associated with it. One of the main things about this system is there isn't very much new data being generated for the system. Instead, what's happening is all the existing data that is already collected about you is basically combined into one big database for each and every person by your ID number. So in China, once you're born, you get an ID number, which is similar to a social security number in the US. We don't really have a similar concept in Germany. And it used to be that your ID number was only necessary for public, like for government stuff, but now you need your ID number for getting a bank account. You need your ID number for buying a cell phone. Even if it's a prepaid cell phone, you still need your ID number. So all your online activity that happens with your cell phone is associated with your ID number, which means you can't really do anything anonymously because it's all going back to your ID number. There's a couple of predecessors, some of them going actually back to the 1990s, that are supposed to be integrated into the new system. One of them or like two of them are blacklists. One of them is a court blacklist. So in China, courts work a little bit differently. They tend to like giving you fines as they do in other countries, but they also like giving you apologies to do. So one of the things, if you do something, for example, you're a company, your food safety wasn't up to par, you have to pay a fine. But in addition to this fine, you also have to write a public apology letter in the newspaper. How you're very sorry that this happened and it won't happen again. And it was a moral failing on your part and it won't happen again. And if you don't do that, you go on this blacklist. Similarly, if you take out a line of credit and don't pay it back within three months or don't do any payments for three months, you go on this debtor's blacklist. If you're on this blacklist, which again is associated with your Shenfeng drone, so your ID number, what happens is you cannot take trains, you cannot take planes. Your life basically becomes very, very inconvenient. Your children can't go to good public schools, your children can't go to private schools, your children can't go to universities. All of these issues are suddenly coming up. There is also a company database that's called Credit China, which is basically similar to the public debtor's blacklist, but it's basically a credit system, a credit score for companies. And then there's the credit reference center of the People's Bank of China, which is a credit score. It was supposed to be like Shufa or like the US FICO for individuals. But one of the big problems in China is that there are a lot of people that aren't part of the formal economy. A lot of people are migrant workers. They get their money in cash. They do not have bank accounts. They do not have anything, they do not have rent or utilities or anything like this because they live in the country, so they own their own home, which they built themselves so they didn't even finance it. And their home isn't officially theirs because in China you can't actually own property. Instead, the government leases it to you. So there were a lot of people that were not covered in this system. And I think the last data that I had was that less than 10% of Chinese adult citizens were actually in the system and had any sort of exposure to banks, which is very, very little. And that meant that people couldn't get credit because banks would only give credit to people that were in the system or people where they had some sort of handling on whether they would be paid back. Now, the implementation details of the new system are very, very scarce. But the basic idea is that Chinese citizens are divided into trustworthy individuals and what the Chinese call trust breakers. Sometimes you have five different groups, sometimes you have two different groups, but in general there's sort of this cut off above this line, it's good and beyond this line, it's bad. This is one graphic from the Wall Street Journal that just shows some of the inputs that go into the system. And one of the things that we see is that the inputs are crazy, crazy varied. So it is, do you pay income taxes? Do you pay your utility bills on time? Do you respect your parents? However they measure that. Do you have a criminal record? Do you pay for public transportation or have you been caught not paying? What about your friends? Do you retweet or use WeChat to distribute sort of information against the party which they call reliability? In actuality it's not about whether it's factual, it's about whether it's against the party or not. Where do you buy and what do you buy? Apparently if you buy diapers it's better than if you buy video games for your score because you know if you buy video games obviously you're not very responsible and if you buy diapers you have a kid you're sort of conforming to the societal idea. And then your score is supposed to go into all these different categories. You're supposed to have better access to social services if your score is good. You're supposed to have better access to internet services. So in theory the idea is that at one point if your score is too bad you're not allowed to use WeChat anymore. You're not allowed to use Alibaba anymore. You can't become a government worker. You cannot take planes and high-speed trains. You cannot get a passport and your insurance premiums will go up. So it's supposed to be this really really big overwhelming system but in actuality what they say their stated goals are is it's a shorthand for a broad range of efforts to improve market security and public safety by increasing integrity and mutual trust in society. So one idea is to allocate resources more efficiently. Resource allocation in China is a pretty big problem because people grow up with there's 1.3 billion people so there's it's always going to be scars and a lot of stuff is people grow up with this idea that it's just very very scars and current distribution strategies which are mostly financially based but also often one-T based don't really seem fair. For example public transport in China is highly subsidized which means that the price does not reflect whether does not reflect true scarcity. So currently the way it works is in theory it's first come first serve and practice there's people that are buying up all the tickets for for example the high-speed train from Shanghai to Beijing and then selling it at a profit or selling it to certain companies that have good ties to the government. That seems very unfair so the new system is supposed to distribute them more fairly and more efficiently. The other thing is restoring trust in people. Perceived interpersonal trust and trust in institutions is extremely low in China. If you're from Germany you might have heard that there's Chinese gangs basically buying up German milk powder and selling it in China. This is actually happening because in 2008 there was a big scandal with laced milk powder and ever since then anyone who can afford it does not use Chinese milk powder because they don't trust the government or the regulations the firms enough to buy Chinese milk powder. So they're actually importing this and the the big irony is sometimes this milk powder is produced in China exported to Germany and then exported back to China. The social credit system is then supposed to identify those that deserve the trust and the third point is sort of a reeducation of people. The idea is they want to make people in the image that the Communist Party thinks people should be and one additional way to the punishments and rewards this could work is the feeling of being surveyed because you can't do anything anonymously. You will automatically adapt your behavior because you know someone is watching you all the time and this is how a lot of the Chinese firewall actually works because most people I know that are sort of more more educated they know ways to circumvent the Chinese firewall but they also know that they're always being watched so they don't do that because you know they're being watched so they self they censor ship they censor themselves. As I've said before allocation of scarce resources so far is mainly through financial or guanxi channels. Guanxi is basically an all-permeeting network of relationships with a clear status hierarchy. So if I attend a school everyone who also attended this school will be sort of in my guanxi network and there's this idea that we will have a system where we are all in group and in group we trust each other and we do favors for each other and everyone who's outside of my immediate group doesn't I don't trust and I don't do favors for and in some ways the guanxi system right now is a substitute for formal institutions in China. For example if you want a passport right now you can of course apply for a passport through regular channels which might take months and months or you can apply for a passport through knowing someone and knowing someone which might take only two days whereas in Germany you have these very regular formal institutions in China they still use guanxi but increasingly especially young people find that guanxi are very unfair because a lot of these are who are where you went to school which is determined by where you're born who your parents are and all these things. Another thing that's important to understand because the system works through public shaming and in a lot of western society we can't really imagine that like I wouldn't really care if my name was in a newspaper of someone who jaywalked for example it would be oh well that's okay but in China this is actually a very very serious thing so saving face is very very important in China and when I went to school there I actually we had this dormitory and it was a all foreigners dormitory where the staff that were responsible for the dormitory felt that foreigners were not behaving in the way they should so their idea was to put the names the pictures and the offenses of the foreigners in the elevator to shame them publicly so for example if you brought a person of the opposite sex to your room they would put your name your offense and your room number in the elevator and of course this didn't work because for a lot of western people it was basically like oh well I'm going to try to be there as often as possible because this is like a batch of honor for me and the Chinese people they figured well this is really really shame and I'm losing my face she brought alcohol so this didn't really work at all but this is kind of the mindset that is behind a lot of these initiatives as I've said there's a lot of problems with we don't really know what's going to happen and one of the ways that we can see what might happen is actually to look at pilot systems China has or like the ever since the communist party took hold that the Chinese government has tried a lot of policy experimentation so whenever they try a new policy they don't roll it out all over but they choose different pilot cities or pilot districts and then they choose oh well this is the district where I'm going to be trying this system and I'm going to be trying another system in another district or city and this is also what they did for the what they're doing for the social credit system now I have three systems that I looked at intensively for this presentation overall there's about 70 that I know of the swimming system swimming is a city in in China the wrong trunk system another city in China and season credit season credit is a commercial system from Alibaba I assume everyone knows Alibaba they're basically the Chinese Amazon except they're bigger and have more users and make more money actually and they have their own little system one of the problems with this kind of system that I found when I tried modeling it was that it's a very very complex system and small changes in input actually changed the output significantly so when they try usually when they try this pilot system they basically have a couple of pilots then they choose the pilot that is best and they roll it out all over but for this kind of thing where you have a lot of complex issues it might not be the best way to do that the swimming system is actually considered the predecessor of all current systems it had a focus on punishment and it was quite interesting at the beginning of the trial period they published a catalog of scores and consequences here is an example this is basically taken from this catalog so if you took out bank loans and didn't repay them you've got deducted 50 points everyone started with a thousand points for this system if you didn't pay pay back your credit cards you also got deducted 50 points if you evaded taxes also 50 points if you sold fake goods 35 points were deducted and actually the system was abolished I think in 2015 2016 because all the the Chinese state media and also a lot of internet citizens talked about how it's an Awellian system and how it's not a good system because it's all very centralized and everything that you do is basically recorded centrally but Krimers writes nonetheless the swimming system already contained the embryonic forms of several elements of subsequent social credit initiatives the notion of dispersional disincentives against rule breaking public naming and shaming of wrongdoers and most importantly the expansion of credit mechanism outside of the market economic context also encompassing compliance with administrative regulation and urban management rules so one of the things that is difficult for especially German speakers is that credit in Chinese Xinyou means credit as in loan but also means credit as in trust so the social credit system is one way of trying to conflict those two the economic credit and the trust credit into one big system but the swimming system basically failed so they adapted the system and are now practicing a new kind of system the wrong term system whenever you read a newspaper article on the social credit system in the west most people went to wrong term because they just received a couple of awards from the Chinese government for being so advanced at this social credit thing but it's it's very difficult to call this one system because there's actually many many intertwined system there is one city level system where city level offenses are recorded for example tax evasion and there's a couple of there's a couple of rules if you evade taxes your score goes down 50 but then if you live in one neighborhood your score might go up for volunteering with the elderly if you live in another neighborhood your score might go up for for example planting some trees in your garden or backyard so depending on your name of your score might be different if you work for a if you work for a taxi cab company for example they also have their own little score system and your score might go up if you get good reviews from your drive from your passengers your score might go down if you don't follow traffic rules these kinds of things there are and there are designated scorekeepers at each level so each district chooses a couple of people who are responsible for passing on the information to the next higher level about who did what there is supposed to supposed to be an official appeals procedure so whenever your score changes you're supposed to be notified but apparently that's not happening at this point for most people um again it's a system of data sharing and one thing that they haven't really disclosed yet is what kind of data is shared are they only sharing the points so if i'm in a district and i plant some trees does the central system get the information person a planted some trees or does the central system get the information person a got five points we don't know at this point and it would mean something very different for how the system could be used but still the end result at this point is that there's one score so you have one central score and it's kind of there's all these different smaller systems that go into this score but at the end everyone has one central score and currently about 85 percent of people are between 950 and 1050 so you start up you start off with 1000 and those are basically the normal people and then anyone above 1050 is considered a trust a trustworthy person and anyone below 1050 is considered a trust breaker and as i've said before with the naming and shaming and all these things what you can actually see here is a billboard with the best trustworthy families in wrong throne so these are the families that have the highest scores for example season credit is a little different it's the only system that actually uses machine learning and artificial intelligence to determine the outputs in wrong chung for example they have artificial intelligence they have computer vision for the most part and the computer vision cameras they decide they try to recognize you when you jaywalk and then when they recognize you when jaywalking you get a small sms well we just saw you jaywalking your score is now dropping but how far how the score develops depending on your jaywalking isn't really determined by machine learning or artificial intelligence instead it's determined by rules you know well one time jaywalking deducts five points and this is stated somewhere um season credit doesn't work like that instead it uses a secret algorithm and the way i talked to some people that work for season credit or for alibaba and the way they described it was they basically clustered their clustered people based on behavior then gave scores to these clusters and then afterwards um did basically reverse engineered their own score using machine learning so that whenever something new happens you can move to a different cluster um this season credit was actually refused accreditation as a as a credit score um in 2017 so banks are not allowed to use the season credit score for your um to use the season credit score to determine whether they give you loans or not because season credit is quite ingenious obviously alibaba wants to keep you within their platform so if you buy using alibaba and using alipay your score goes up if you buy using wechat pay which is a competing platform your score goes down and this uses many of the same rewards mechanism of the official government systems and this is just an illustration of what kind of scores you can have apparently your scores can go between 350 and 850 and in chinese there's basically five different levels so 385 is um trust breaker or missing trust and then 731 is trust break trust is exceedingly high so um one way i tried to approach this issue was through agent-based modeling social credit system is individual level but what we are really interested in or what i'm really interested in is actually societal level consequences so if everyone gets this score what does that mean for society um agent-based modeling works quite well for that because it allows us to imbue agents with some sort of rationality but with a bounded rationality what does bounded rationality mean usually in economics people assume agents are completely rational so they are profit maximizers they have all the information but in reality agents don't have all the informations they have a lot of issues with um issues with keeping stuff in their mind so a lot of the time they won't choose the best thing in the world but they choose the best thing that they see um and bounded rationality allows us to account for this thing it allows us to account for um heuristics and these things and what i did is i took the propensity for specific behavior from current state of the art research mostly from behavioral economics for example um i looked at tax evasion and i looked at who is likely to evade taxes in a in a system and then obviously there was some stochastic some chance element but the distribution that i chose is um related to the current research and i also checked that um my model has similar results to the wrong-tongue model which i modeled at at the beginning um so on average 87 percent um of my users have a score of within 10 percent of the original score which is also the data that um wrong-tongue city actually publishes now um for the most part i compared four different i compared design choices in two axes one of them was a centralized system versus a multi-level system and a rule-based system versus versus a machine learning system the centralized system is basically you have a central you have um all the information is kept centrally and um everyone in china or wherever in wrong-tongue has the exact same scoring opportunities um now if you have a centralized system the clear expectations were um were pretty good but at the same time the acceptance from the population was really really low um which they found during the swimming experiment and there's also the problem of a single point of failure who decides the central catalog and um depending on who sort of has the power it kind of just um reproduces power structures so because you have this central catalog the people that are in power centrally they are basically deciding some sort of score mechanism that works for them very well so that they and their family will have high scores a multi-level system um has the advantage that local adaptation kind of works and there's sort of many points of failure but um in my model when I allowed locals to basically set their own rules what happened was that they competed so it started out being this district of wrong-tongue for example and this district of wrong-tongue they compete for the best people that they want to attract and suddenly you have this kind of race to the bottom where um people want to move where they wouldn't be prosecuted so they move to places where there's less cameras for example at the same time there's many points of failures especially the way it's currently set up with um people reporting data to the next to the next high level and a lot of the time what we have actually seen in wrong-tongue was that they reported data on people they didn't like more than data on people they did like or their families got better scores than people they didn't know so it also kind of um reproduced these biases the um rule-based system has the advantage that um people were more prone to adapt their behavior because they actually knew what they needed to do in order to adapt their behavior but the score didn't really correlate with the important characteristics that they actually cared about and um as opposed to if in this machine learning system you know how in Germany we don't really know the shufa algorithm and I for example don't exactly know what I could do in order to improve my shufa score and this is a similar system in China with the sesam credit score a lot of people don't really they they say well I really want to adapt my behavior to the score to improve my score but when I tried doing that my behavior my score actually got worse and you can have different biases that I'm going to be talking about in a little bit um there's also this big problem of incentive mismatch so um the decentralized rules-based systems like rongchong which is the system that I analyze the most why because I believe this is the system that we're moving towards right now because rongchong won a lot of awards so the Chinese government the way they usually work is they try pilots then they choose the best couple of systems they give them awards and then they they roll out the system nationwide so I assume that the system that's going to be the system in the end will be similar to the rongchong system now one problem that I actually saw in my um simulation was that you could have this possible race to the bottom um there's also this conflict of interest in those that set the rules because a lot of the time the way it works is you have come you have your company in your company you in in combination with your um party leaders actually decide on the rules for your um for the score system but the scores of all your employees actually determines your company's score if you employ a lot of people with high scores you get a better score so you have this incentive to give out high scores and to make sure that everyone gets high scores but at the same time the government has an incentive for scores to be comparable um so there's a lot of int incentives mismatch the government also has the incentive to keep false negatives down but they actually the way the Chinese system currently works is um they emphasize catching trust breakers more than rewarding trust follow or like trustworthy people so um false positives for them are less important but false positives erode the trust in the system and they lead to a lot less behavioral adaptation I was actually able to show this using some um nudging research where that showed that as soon as you introduce a couple an error probability and you can be caught for something that you didn't do your your probability of um changing your behavior based on the score are actually is actually lower and in Rongchang one of the perverse things that they're doing is you can um you can donate money to the party or to party to like party affiliated um social services and this will give you points um which is kind of an indulgence system which is quite interesting especially because a lot of these donation systems work that way work in a way that you can donate 50 000 renminbi and you get 50 points and then you donate another 50 000 renminbi and you get another 50 points so you can basically donate a lot of money and then behave however you want and still get a good score and um the trust in other people can actually go down even more in the system because suddenly you only trust them because of their scores and um the current system is set up so that you can actually look up scores of everyone that you want to work with and if they don't have a score high enough then suddenly you don't want to work with them um the trust in the legal system can also decrease actually um why because trust in the legal system in china is already low and a lot of the things like jaywalking they're already illegal in china as they are here but no one cares and suddenly you have this parallel system that um punishes you for whatever but why do you why don't you just try to fix the legal system which would be my approach you can suddenly illegal activity could happen more offline and this is one of those things that um is quite interesting in countries that we've seen that have moved towards mobile payments and away from cash you see less robberies but so you don't see less um you don't actually see less crime instead you see more new types of crime so you see more credit card fraud you see more um phone robberies these kinds of things and this is also where um things could move for um that in the chinese case one major problem is also that this new system i've talked a little bit about this one but it can introduce a lot of new bias and reproduce the bias even more so um for example um china is a country of 55 minorities the hun are a big majority they have about 94 percent of the population so any computer vision task we've shown that they are really really bad at discriminating or at discriminating between individuals in um smaller ethnic groups in the u.s most computer vision tasks perform worse for african americans they perform worse for women because all of the training sets are male and white and maybe asian in china all of these tasks are actually um performing worse for ethnic minorities for the yuigurs for example and one way that they could try to abuse the system is to basically just what they also what they're also doing already in qinjiang is to basically just identify oh this is a person of the minority well i'm just gonna go i'm just gonna go and check him or her more more thoroughly this is actually what happens in qinjiang if you're in qinjiang and you look like a turkish person or like from turkish people um you can you are a lot more likely to be questions you're a lot more likely to be stopped and they ask you or require you to download spyware on your phone um and this is currently what happens and this new kind of system can actually help you with that um i've said that it can reproduce these kind of power structures and now obviously we all know newtville technology doesn't really exist but in the chinese case in the social credit case they don't even pretend they always say well this is newtville technology and it's all a lot better but actually it's the people currently in power they decide on what gives what gives you point and what detect what deducts points for you at the problem currently the the entire system is set up in a way that it's all goes together with your shan fendrong with your id card what if you don't have an id card that's foreigners for one but it's also people in china that were born during the one child policy and were not registered there's quite a lot of them actually they're not registered anywhere and suddenly they can't do anything because they don't have a score they can't get a phone they can't do anything really and um part of the part of the push with this um social credit system is to go away from cash actually so if you you need to use your phone to pay but for your phone you need your shan fendrong if you don't have a shan fendrong well tough luck for you and um currently the system in wrong trung is set up in a way that you can check other people's scores and you can also see what they lose points for so you can actually sort of choose to discriminate against people that are gay for example because they might have lost points for going to a gay bar which you can lose points for another big issue currently is data privacy and security um personal data is grossly undervalued in china if you ask a chinese person what do you think how much is your data worth they're going to be saying what data i don't have data um and currently the way it works is if you have someone's id number which is quite easy to find out um you can actually buy access to a lot of personal information for a small fee so you pay about a hundred euros and you get um all hotel bookings of the last year you get information of who booked these hotels with them you get information of where they stayed you get trained bookings you get access to all of the official databases for this one person and for another 700 rendman b you can actually get live location data so you can get the data of where this person is right now or where he's on her phone is right now but if you've ever been to china you know that where the phone is usually the people aren't far um soup china actually did an experiment where a couple of journalists tried buying that because it's actually these kind of services are offered on we chant pretty publicly and you can just buy them quite easily um so one additional thing that i looked at is um because one of the things that is quite interesting is you have this idea of credit as twofold credit is um trust credit but credit is also low in credit and what if credit institutions actually use this unified credit score to determine credit distribution the idea is that it's supposed to lead to reduced information asymmetry obviously so fewer defaults and overall more credit creation new people are supposed to get access to credit and they're supposed to be less shadow banking but what actually happens i'm not going to be talking about how i set up the model but just about my results if you have this kind of score that includes credit information but also includes morally good or measures of being morally good what you have is in the beginning about 30 more agents get access to credit and especially people that previously have not gotten credit access suddenly have credit access but the problem is that this social credit score that correlates all of these different issues um it correlates only very very weakly with repayment ability or repayment wishes and thus suddenly you have all of these non-performing loans um you have and what we see is sort of like we have non-performing loans banks give out less loans because they have so many non-performing loans and um then the non-performing loans are written off and suddenly banks give out more loans but you have this oscillating financial system where you have where you give out a lot of loans a lot of them are non-performing then you give out a lot of loans again and this is very very vulnerable to crisis if you have a real economic crisis during the time where non-performing loans are high then a lot of banks will actually default which is very very dangerous for a financial system as nationed as the Chinese one now what are some possible corrections you could create a score that um basically is the same as a shoofa score so that looks only at credit decisions but suddenly you lose a lot of incentives for the social credit score if the social credit score doesn't matter for credit distribution anymore another ish thing and this is i think the more likely one is that you have a blacklist for people that have not repaid a loan in the past so you basically get one freebie and afterwards if you didn't repay a loan in the past then you will not get a loan in the future you will still be part of the social credit system and your social credit score will still be important for all of these after-access issues but it won't be important for access to loans anymore once you've been on this blacklist which is probably something that the chinese government could go behind but it's also more more effort to take care of it then you have to think about well you can't leave them on the blacklist forever so how long do you leave them on the blacklist do they have to pay back the loan and then they get off the blacklist or do they have to pay back the loan and then stay and then like stay not not in default for a year or for five years there's a lot of so small decisions that in my opinion the chinese government hasn't really thought about up until now because they they're basically doing all these pilot studies and all of these regional governments are thinking of all these small things but they're not documenting everything that they're doing so once they they want to roll it out by 2020 by the way nationwide once they've already out there's a pretty big chance in my opinion that they have a lot of unintended consequences a lot of things that they haven't thought about and that they will then have to look at we also um yeah so i believe that some sort of system is likely to come just in terms of how much energy they've expanded into this one and for the chinese government at this point for the party it would be losing face if they did not include any such system because they've been talking about this for a while but most likely it would be a kind of decentralized data sharing system and when i ran my simulation by the way i will um make public my code i still need some um basically i used some proprietary data for my model and i still need the the permission to publish this once i publish this one i will also tweet it and we'll put it on github for everyone to play around with if you want to and one of some of these implementation details that were very important in determining model outcomes were do do we have a relative or absolute ranking so far all of the system i looked at had absolute rankings but there is a point to be made for relative rankings do we have one score where basically if you're a chinese person you get one score or do we have different sub scores in different fields do we have people reporting behavior or do we have automatic behavior recording how do you access other people's scores how much information can you get from other people's scores currently if someone is on a blacklist for example if you have their id number again you can put it into this blacklist and then they will say oh this person is on this blacklist for not following this judges order and then it says what kind of judges order it was so most likely it will be it will be something like this the idea is that this the social credit system isn't only for individuals but also for firms and for NGOs so what kind of roles will firms play in this system i haven't looked at that in detail at this point but it will be very interesting another idea that western people often talk about is do people also rank each other currently that's not part of the system in china but it might be at one point and lastly where does the aggregation happen so we've i said that a lot of it is actually data sharing in china so um what kind of data is shared do is the raw data shared person a did something or is the aggregated data shared person a got this score at this point most of the time it is actually the raw data that is shared but that also has sort of these data privacy issues of course that i've talked about okay perfect now there's 10 more minutes thank you for your attention if you have questions remarks you can ask them now you can catch me up later you can tweet to me or send me an email whatever you're interested in thank you very much if you're watching this please ask your question to ios here on twitter we'll also try to make sure to get to those let's just go ahead and start with mic one thank you very much for this beautiful talk um i was wondering how did the chinese government companies and most of all citizens themselves respond to you doing this research or let's put it differently if you would have been in the system yourself how would your research affect your social credit score um so there's there's actually two different two different responses that i've seen when i talk to the government themselves because i was there in the government scholarship and mentioned that i'm really interested in this they basically said oh well this is just a technical system you don't really need to be concerned with this it's not very important just you know it's just a technicality it's just for us to make life more efficient and better for everyone so i assume my score would actually go down from doing this research actually but when i talk to a lot of people at universities they were also very they were very interested in my research and a lot of them mentioned that they didn't even know that the system existed thank you very much before we go to a question from our signal angel a request for all the people leaving the room please do so as quietly as possible so we can continue its q and a um the signal angel please yeah the jedix wants to know um is the score actually influenced by association with people with the lowest score meaning that uh is there any peer pressure to stay away from people with bad scores um the season credit score definitely is influenced by your friend's scores the um wrong chung score so far apparently is not influenced but it is definitely in the cards and is it is planned that it will be part of this i think um the we we chat which is the main platform it's sort of like whatsapp exactly can do a lot a lot more we chat um is still not connected to the social credit score in wrong chung once they do that it will most likely also reflect your score all right let's continue with mic 3 i have a question about your model so i'm wondering um what kind of interactions are you modeling or actions like what can the agents actually do you mention moving somewhere else and um what else okay so the way i set up my model was um i set up a multi-level model so i looked at um different kinds of levels i started out with basically they can evade taxes they can get low ends and repay low ends um they can choose where to live um and they can um follow traffic rules or not follow traffic rules and um because these were sort of four big issues that were mentioned in all of the different systems so i started out with these issues and looked at um what kind of behavior do i see i used some research that um some friends of mine actually sent out surveys to people and asked them well you're now part of the system did did your behavior change and how did it change depending on your responses um depending on your score and depending on the score system that exists and i basically used that and some other research on nudging and on behavioral adaptation to look at um how likely is it that someone would change their behavior based on the score all right let's do another question from the interwebs yeah it's actually two questions in one um how does this system work for chinese people living abroad or for non-citizens that do business in china um currently the system does not work for non-citizens that do business in china because it works through the shan fan drang you only get a shan fan drang if you're a chinese citizen or you live in china for 10 or more years so everyone who is not chinese is currently excluded chinese people not living in china if they have a shan fan drang are on this system but um there's not a lot of information all right uh mic 4 well we've come a long way since the port city's world title um can you tell us anything about the dynamic in the time dimension uh how quickly can i regain credit that was lost i would be at any observations there um so in the suining system what they actually did was they had a very very strict period so if you evaded taxes your score would be down for two years and then it would rebalance in the rung chung system they did not publish this kind of period so my assumption is that it's going to be more on a case by case basis because i looked at the chinese data i looked at the chinese policy documents um and they didn't really for most of the stuff they didn't say how long it would count for the blacklists which was kind of the predecessor that we look at currently the way it works is you stay on there until whatever the black until whatever the reason for the blacklist is has been resolved so you stay on there until you send off this apology that the judge ordered you to and then usually you still needed to apply to get off so it doesn't for blacklist it does not work that you automatically get off you need to apply you need to show that you've done what they ask you to do and then you can get off this blacklist and i assume it will be a similar sort of appeals procedure for the system all right let's go to mic 2 thank you i just wanted to ask if looking up someone else's data in detail like position etc does affect your own score currently it apparently does not or at least they haven't published that it does it might in the future but most likely it's actually behavior that they want so they want you to look up other people's scores before doing business with them they want you to basically use this to decide who you're going to associate with thank you all right do we have another question from the internet maybe yes i do stand by the question is how is this actually implemented for the offline rural population in china quite easily not at all at this point the idea is by 2020 that they will actually have all of this implemented but even for the offline or let's say offline rural population in china is getting smaller and smaller even in rural villages you have about 50 60 percent of people that are online that and most of them are online via smartphone and the smartphone is connected to the shunfeng drum so it's not very complicated to do that for everyone who's online for everyone who's offline of course this is more problematic but i think the end goal is to not have people offline at all all right let's jump right back to the microphone too please okay thank you for the very good and frightening talk so far i at first i have to correct you in one point in germany we have a similar system because we have this tax id which is set from birth on and rests 30 years after a person's death um yeah so we have a lifelong id you're right i just i don't know mine so i figured no problem but at least we could establish a similar system if we if we have a government which would want it um the second question a second question from me with um you mentioned this quanchi quanchi is it a kind of a social network i didn't understand it really yes it is a kind of social network but one that is um a lot more based on hierarchies than it is in the west so you have you have people that are above you and people that are below you and the expectation is that while it's a quid for a crow people that are above you in the hierarchy will give you less than you will give to them aha okay okay all right um unfortunately we are out of time so please give another huge applause for tony
that you always hear about in Western media interesting to look at the social credit system of the plan. The Chinese government, they say in China is that there are a lot of people that not have anything, they do not have rent or more efficiently. Resource allocation in China is very unfair so the new system is supposed to additional way to the punishments and rewards know that they're always being watched so they scarce resources so far is mainly through idea that we will have a system where we are all pilot systems China has or like the ever since mean something very different for how the system agent-based modeling social credit system is so that they and their family will have high advantage that um people were more prone to adapt you're a lot more likely to be stopped and they create a score that um basically is the same as a it out by 2020 by the way nationwide once they've some of these implementation details that were is do people also rank each other currently chinese government companies and most of all