Investigating Social Dynamics in the Digital Era: A Data-driven Approach
Zollo, Fabiana
The COVID-19 pandemic has highlighted the challenge of conveying and communicating complexity and uncertainty to the public, also given the increasing central role of the Internet and social media. Designed to maximise users' presence on the platform and to deliver targeted advertising, social media transformed the information landscape and have rapidly become the main information sources for many of their users. Information spreads faster and farther online, in a flow-through system where users have immediate access to unlimited content. This may facilitate the proliferation of mis- and dis-information, generating chaos, and limiting access to correct information. In this talk, I will provide an overview of how online social dynamics and behavioural patterns can be investigated in a quantitative and interdisciplinary way. Moreover, I will discuss how data-driven insights can be used to design tailored policy recommendations.
 I will present very quickly some of the previous results that we had working with online data and applying a data-driven approach to study social dynamics with online social media. Then in the second part of this presentation, I will focus on some ongoing projects and some recent results on polarization and also on the link between polarization and speech. I would like to start from the digital information space. I work with mainly online data and with data from social media platforms. With the event of the internet and later with social media, we have lots of information that is being produced by users, so by us, when we use all our devices in our digital lives, and also the fact that this information is a lot. We produce a lot of information that, of course, can be precious if we want to run some kind of analysis, but on the other side, the fact that we produce this information and that information is there is also something that we, as users, have to process. The idea that we have lots of information that is going around on the web and that can spread very easily and very quickly. This was a great advantage and revolutionized the way in which we communicate with each other. The fact that a piece of information can spread, so we have this first key point of this space. The fact that information can spread very easily and very quickly and the fact that we have lots of information to process. Also, the fact that this space, the digital information space, changed if we compare this space to the traditional media system. The main difference is that while in the traditional media system, the users, let's say, on the other side are passive, so they do not have a direct, let's say, interaction with the information source. This is not happening on the internet and, of course, it's not happening on social media, where the users play an active role in not only accessing content and selecting the disinformation, but also in producing their own information. This is another key point because we also observed the emergence of a more heterogeneous system of new sources. For sure, this can also be an advantage. It can, for example, foster a pluralism from a certain perspective. But what we also observe is that things are a bit more complicated. What we observed in analyzing this, so what we do, just to quickly describe you what we do is to collect and analyze data from online media platforms. We use methods from computational social science, so we use methods from mathematics, statistics, computer science, to analyze this data. But usually what we do is to answer questions that are questions sometimes from the social sciences. It's a different approach. I would say a complementary approach. It cannot totally substitute the traditional social sciences approach, but we can have sometimes a broader overview of what's happening. Analyzing this data, what we have seen is that the online digital system is characterized by strong polarization. We did lots of studies on different topics from climate change to vaccines, to also science and pseudoscience, and in all cases where the controversial issue is debated, the users polarized. You have users that just got informed on a certain part, certain methods, a category of new sources, and users on the other side of the spectrum. Although we can say that we could have access to lots of information and information sources, what we observe is that in the end, the users tend to confine their attention to well segregated groups of like-minded individuals. What I am referring to are the now very famous eco chambers. The fact that we can observe the emergence of this, we call them clusters in computer science. They are groups of individuals where you have lots of internal connections. It's a very dense region in the network, but the connections between the other groups with the other groups are not so many. The fact that you choose to interact with a certain kind of information means that you are confining your attention to that narrative, to that particular position, for example, on the topic, and you are not interacting and so having, for example, a debate with the other side, with the other position. This can be, let's say, a problem in the sense that it can reduce pluralism, for example, because what we also observed is not only that the debate is very polarized and that these groups of like-minded individuals emerge, but also we observed the selective exposure. The fact that even on more general, so if we analyze all the information, for example, we had this study on more than 360 million users on English sources on a global scale on Facebook. Also in that case, we found these communities, these eco chambers were emerging. What we also observed was that users were, so the number of new sources we interact with decreases with the increase of our activity on the social media platform or with the increase of the time that our range would say on the platform. So this means that, although potentially, we have access to this great amount of information, what's happening in the end is that we are really confining our attention and so we are selecting information from a very small set of new sources. These dynamics are, of course, valid for information in general, but if we just think of misinformation as a particular kind of information with specific characteristics, so for example, that it's not reliable, you can understand that the mechanisms that are behind are the same. So in this direction, it's very important to understand how these dynamics can also influence how misinformation, for example, spreads and can spread and what are the factors that are behind this spreading. What we also observed, for example, we studied the dynamics and the emotional dynamics of the users involved in these communities and what we have seen is that, for example, the negative emotional attitude of users increases with their polarization, but especially that the debate degenerates when you have users from one community to interact and discuss with users from the opposing community. So this is also showing us that we have this very polarized environment, but what we also have is that users from different eco chambers are not only very rarely interacting, but they're also, when they interact, most of the times this interaction is not a civil discussion, a civil debate, but it's a debate that degenerates over time. So this again can reduce the possibility of dialogue and debate, for example, in a democracy. So what are the also the implication of this kind of system is that what we also did was to study how, for example, the debunking content that was created on Facebook to debunk false claims and hoaxes, how this content was received by, for example, pseudo scientific communities. And what we observed was that this content, because of the eco chambers structure, was not really reaching those communities. So we observed that the main consumers of this kind of content are users from the scientific eco chambers, so users that we would expect not, you know, needing this kind of corrections. And so, and this is the first point. So the fact that having this really polarized environment and this structure in eco chambers can also limit sometimes the contrasting approaches that we can implement. On the other side, what we also observed was that in the cases where debunking was able to reach the target, let's say, in those cases, we observed what we can define a backfire effect because these users from the pseudo scientific community reinforced their attachment and their involvement in the pseudo scientific community after being in contact with the debunking correction. So it was not from our data what we were observing. So this kind of approach on social media was not really, was a bit ineffective. And what I would like to say is that if you think about what I was saying before, when we have a system like this where information gets spread very easily and very and very fastly, the idea of debunking every single piece of content that is not reliable is also feasible from a human point of view. And so what we tried to do was to try to understand how, so what kind of approaches, what we can do to try to contrast misinformation from a different point of view. So not acting when misinformation is out there, but try to do something before. And what I would like to present today, so I will first talk about a paper work that we did for the early warning, I will explain in a minute what I mean. And then I'll represent some of the most recent projects I've been working on. So giving you some of, it's an idea of how we can work with this kind of data. So what we observed looking at the data was that when we have a certain topic appearing on official news, we have a sort of 24 hours window in which this topic can appear on fake news. So the idea is that we don't have much time because it's only 24 hour window, but we have some time maybe to make an intervention. And the idea of the intervention was that, so if I'm able, let's say to predict what kind of topics, what kind of entities will become subject, are likely, let's say to become subject of fake news in the short term, what I can do is that I can try to adjust the communication and on these topics. Because what we observed in all our studies was that we have this very strong link between misinformation and polarisation. Lots of the times misinformation, especially disinformation, so when it's created a doc, it's a kind of information that leverages the polarisation dynamics. And so what we did in this study was to understand if looking, not at the content, so not at the, not trying to, you know, define if the content is fake news or not, but try to understand how users are interacting with content. If this information give us some hints to understand what's happening in the short, in the short time, so in the future. So what we did was to, we developed a framework for raising an early warning of potential misinformation targets. So the idea of this framework was to being able to predict what topics are likely to be subject of misinformation in the short term. So the idea, in the image that we have a system like this, for example, in press rooms, what the journalist on the other side knows is that maybe what he can do is that adjust the communication strategy on that particular topic so that he can try to reduce polarisation around that topic and limit the spreading or the potential spreading on misinformation on that topic. So what we did in this, what it is, what we did in this work is pretty technical in the sense that we had the classifier, we developed a classifier to being able to raise this warning. But what I want just to highlight here is the fact that this classifier is just using information from how users are interacting with content, not the content itself. So looking at, for example, the sentiment of the users involved in the discussion, the distance between how the, for example, the content is presented, how the content is perceived, or how the content is perceived between, so in different communities. So all these kinds of features were useful to produce a classifier that was able to make this kind of distinction. So now imagine that we have a system like this. Now the question is what we can do with this kind of system because, okay, I know that I have some topics, for example, that can become, can become object of fake news, okay. What's the communication strategy that I should adopt to try to understand, so to try to reduce that polarization because that was the aim of the, so with the idea of I reduce the polarization and the discussion on the topic is less polarized, maybe more civil. And so what happens is that the topic is not, is not, is not something that can be of interest for the misinformation content. So we did the first, let's say, experiment of this, working with the Lamos School of Economics and Corriere la Siera. Corriere la Siera is the main newspaper that we have in Italy. The idea of this project was to try to understand how the way in which you communicate the news impacted the way in which the users shape the debate around that particular, that particular news. We analyzed in this project data from Facebook and from Twitter and we monitored the reaction of readers to fact-based journalistic outputs. In this project, which was the issue of migration, this was, so this is a project from, that started in 2017. The migration issue was one of the hot topics in Italy, Italy's agenda. And it was a very interesting debate, very polarized debate in our country. So it was a good example to understand how we could try to, to shape the debate, to reduce the polarization around the debate. What was interesting about the migration was also that we didn't have really an emergency in Italy about migration. So the issue was stopping the newspaper's agenda, but we were observing a strong decrease in your rivals in Italy. So, but the topic was very present also on social media. And so it was also interesting to understand this kind of dynamics. So people were really interested in a topic that it was not an emergency. And what was also, of course, it was a topic that was greatly discussed at political level. And what, for example, we observed, and this was very interesting, what you are seeing here on this graph, what we can define at the eco chambers. If you look on the left side of this graph, you have on the circle, on the ring, what you have are all the new sources that we have in Italy. And they have a different color according to the community that belong to. So, and there is an edge, so a connection between two different new sources if they have at least one user in common. So, using this information, we were able to extract these three main communities. And these three main communities can be referred to the main, to we can, the light blue one is what we call the mainstream media. Then we had the right in the right in orange and in blue gray, we have the five star movement, let's say the populism, the populist parties in Italy. What's very interesting is that usually in our works, these kind of eco chambers are pretty stable over time. So they do not really change, okay, if you observe what's happening over time, what we observed here was that in June 2018, we had a need for the formation of a coalition government between the right and the five star movement. And what we observed is that almost instantaneously, these two groups merged in one group. So what you do, what you see in the following months is that from June to August, what we had was the mainstream community and the orange is now representing the right, so the merging between the right and the five star movement. So this is also explaining the fact that these communities really emerge because of the, you know, selection of some information from users. So the fact that now those new sources were together in the coalition government. And so the narrative about migration was pretty similar between the two parties reflected in the way in which also the news were communicating and the users were interacting with this information. But what we did, so this was interesting to analyze the dynamics, but what we did was really to perform an experiment where the journalist had to produce some stories on the topic of migration and they shared these stories on Facebook. And the journalists, of course, were completely free to choose the content of the news based on what was happening. But what we asked the journalist to do was to try to use different content types and techniques when they were communicating the news. So for example, to use infographics or plain text or to use to write an editorial or news reports or human interest stories, so stories where they were telling the story of the people involved in the news. And so for example, why they arrived in Italy, what they had to deal with before coming here and so on. And we trapped all the data from the Facebook page in terms of engagement. We also measured the sentiment of the comments of the users. And we had a group of human annotators to annotate more than 26,000 comments, to assign three labels. The toxic label, so to understand if the comment was aggressive or something that would let the user abandon a conversation. And we used this label as a proxy for the civil discourse, mistrust of Corriere, so as a proxy of trust of the news source. If for example, in the comment, we could have a direct attack to the credibility of the source of information and the position on migration. So if the user was in favor or against the migration phenomenon. And we used all this data to make some analysis and to understand what was working better than other source techniques or types to smooth polarization and facilitate a civil debate among the readers. And what we observed was that, for example, all multimedia pieces and especially pieces that had videos inside were pretty, so were very, received very strong and very supportive engagement with a higher number of likes and shares and were very rarely criticized. We think that this can also depend on the fact that having this kind of content, so for example pictures or videos in some sense makes more difficult for users to go against the source. But on the other side, where all the content types where the journalists were pushing facts to the forefront, so like for example infographics, in this case, we had lots more debate and discussion and lots of pushback. So some sort of backfire aspect that we can also see here and the high criticism of the media. This was something that we were expecting. So the fact that having this kind of approach of a explicit direct sometimes correction against a very sometimes emotional topic like the position on migration couldn't work in the end. And for example, for the techniques, all the news reports that were more, let's say straightforward, impartial, emotional, where the type of news that elicited the most trust, but the human interest stories got a very strong pushback. This was not something that the journalists were expecting because the idea of these stories, as I said before, was really to sometimes also have a sort of more empathic approach. So to try to let people understand what was really happening to these people and what were the reasons why these people were coming to Italy. But on the other side, it can be that these kind of stories were felt as something, you know, the user felt manipulated into taking a certain side because of this more emotional news from the journalists. And in this case, we observed lots of toxic comments, lots of criticism, lots of anti-migration comments. And in general, every time that the journalist provided a very strong opinion of, for example, a policy proposal, like in the case of constructive journalism, where they were proposing a solution, for example, to the problem, there was a lot of criticism of the source. So the career was not trust by the commenters and lots of anti-migration comments. So in general, using this data-driven approach, so like the infographics or giving, you know, numbers and data to support the facts, was not really happening to reduce the polarization of the debate. So of course, what I want to say is that these were interesting insights. Of course, they're related to a specific topic. They're also related to our context in Italy. And of course, there can also be an influence or from the same way in which the algorithms work, for example, in trying to push some kind of content, like, for example, the multimedia content over other kinds of content. But this was interesting also to understand that the way in which you do your communication, so the way in which you decide to communicate the same piece of news, can really change, can really shape how the debate, so the debate on the other side. And so it can also be a way to try to think on how we can contrast, for example, misinformation and polarization, not directly, but changing the approach on the communication side. And speaking on the communication, from the communication side, what I want to mention very quickly is that we also had this European project that was a two-year European project that finished last July, where we analyzed how science is communicating in Europe. We did this for seven countries and for the Twitter, Facebook and YouTube, so three different platforms, and also for three controversial topics. And the results are interesting because they also show differences in how the science communication is communicated, not only in the three platforms, but also across the different countries. And so the fact that, again, when we analyze, for example, social media data, it's also very important to take into account the fact that also how the people and interact with information is also sometimes changing from country to country, where we can, when users display different presences in how they interact with content. If you are interested in this, there is the comprehensive report of this work on available, I have the link here in the slide and on the website of the project. What we did in the project was also to develop some quality indicators for science communication. So what are the aspects that you have to take into account to improve and to ensure that your communication is a quality communication? But what we also did for social media was that we tested a series of recommendations for a period of five months, and then we developed this recommendation for communicating science on social media. This was a pretty interesting, again, experiment because the practices and the recommendations that we developed are data-driven recommendations because we used the data that I was mentioning before to develop these recommendations, together with the quality indicators that were developed through co-design activities and qualitative research. And we put together all this information to develop these recommendations that before being published, were also tested from communication practitioners on social media. And again, all this information and these products are available on the website. So now in the last, the other thing that I want to mention is that we are working also on the infodemic and especially on COVID-19. So for example, we had this work where we analyzed the COVID-19 infodemic on these five platforms and we observed that all these platforms displayed the some, so displayed the really values that can be, so we measured the infodemic in a way similar to how you can measure the epidemic. So we computed the R0 for all these platforms and we observed that the, indeed, these values were hypercritical for all the platforms last year at the beginning of the pandemic. And interesting, what we observed here was also that the way in which misinformation was spreading in these platforms was similar to how reliable information was spreading. So we didn't observe different patterns in how misinformation was spreading around. What we observed was that the ratio of misinformation was higher in platforms where, so not in mainstream platforms, but for example in Gabbo Reddit that are platforms where we have less, let's say, moderation of content. And the last thing that I want to mention, the last project in these 10 minutes I guess I have, is this project on 8-Speech is again a European project which has several goals. So on one side, the detection of 8-Speech in different languages and the identification of triggers and then the recommendation of counter narratives and the proposal of policies to the European communication regulators. We now have, I will now show the first results where we used the classifier for detecting 8-Speech and we did this study trying to understand if there is a link between misinformation and 8-Speech. So the idea is that of course the detection of this kind of, so the automatic detection of this kind of content is complicated and also the contrast is complicated. For example, starting from the same definition, all the platforms have different policies and have different definitions, for example, of what is considered hate speech or what's speech that is not allowed by the platform. So the idea, so what we did in this work was to use a definition of 8-Speech that refers to the whole spectrum of language that can be used in online debates. So our definition includes normal, so from normal to, for example, acceptable, inappropriate or violent speech and violent speech really cover all the forms that spread inside and promote hate and but we also considered, for example, inappropriate or offensive speech. So these kind of speeches are not illegal but they can deteriorate the public debate and they can post radicalization. And we used this data to, so to first to develop the classifier. So we used in this work data from the YouTube platform and the topic in Italian, so Italian from Italian data. The topic was the COVID-19 and we have all this data and we also have channels and videos from that where we can consider reliable and questionable. And what we did was to develop this classification model. So how can you develop this? You take a sample of your comments and you we hired a group of human annotators to annotate these comments across the different labels, so appropriate, inappropriate, offensive, violent. And then when you have this data, you use this data to train the classification, so to train the algorithm so that the algorithm is able to understand, so to evaluate if a certain, so to learn from the human annotations how it can identify comments in the different in the different labels. And of course the model then is evaluated. And when this process is concluded, you have a classifier that works with a certain accuracy. What you do is also that you compare the, so how good your model is against how good the annotators were. So in our case, our model is pretty close to what the annotators, to the agreement between and among the annotators in our, in our class annotation task. And so we can say that we have a very high quality model. Once you have the model, what you do is that you apply the model to the whole data set of comments because of course it's not possible to manually annotate millions of comments. So the idea of using these classificators is that you try to use this, you use the model to classify the rest of the, the rest of the comments. So what we observe is that there is the proportion of the hate speech labels, the forward speech labels in the whole data set are similar for questionable and for reliable channels. So we don't observe any particular differences. And we also measured the comment delay. So the time that elapsed between the posting time of a video, so when a video is posted and the comment that, the first comment that arrived, arrives to the video in hours. And what we see is that there is, so if we look at the different kinds of comments, there is not great difference, but on questionable channels, toxic comments appear first and faster than appropriate ones. And following the decreasing levels of toxicity. So when we are on questionable channels, so channels that we can, where we can have fake news or all, you know, that kind of unreliable information, the toxic comments, so comments that we can define offensive, violent or inappropriate, arrive first and faster than the, than the others. What we also observe, and this is pretty interesting, was that we were not able to observe the presence of what we called the pure haters. So the users that only use a violent language, this was not the case in our data. So when you look at the data, what you see is that we have users that are what we can say, you know, normal users that happen to use a violent, so hate speech in some situations. And indeed, one of the, you know, the future work is devoted to the understanding what kind of triggers, okay, you can have in the, in the, you can have there to try to to foster this kind of behavior. So we have very few users in our data set that only use violent comments, but their activity is pretty low. So they have less than five comments. The other thing that we observed was that, so we computed the, let's say, polarization of these users. So if they were using and getting videos from reliable channel or from questionable channels. And what we observe is that even here we have two peaks, okay, in correspondence of these extreme values of linings. So what we, what we can say is that polarize, more polarized users tend to use a higher proportions of non-appropriate violent toxic comments. But interestingly, the users skewed towards reliable channels, post on average, a higher proportion of non-appropriate comments, then users skewed towards questionable channels. And what's also coming from this data is that these users mainly use inappropriate comments in their opponents community. So again, if you remember what I was saying at the beginning about the, how these, the, you know, interaction dynamics take form and the fact that the debate is degenerating, we are confirming what we were observing and the level of sentiment, even looking at hate speech. And what we observe is also in this case that the toxicity level, so the average of the toxicity values correlates possibly with the length of the discussion. So the more the discussion, so when the discussion increases in terms of comments and in terms of time, the toxicity of this discussion increases. So we have a more toxic debate when the discussion gets longer. So, yeah, this is just a summary of the fact that we didn't find any evidence of relationship between the use of a toxic, toxic language and being involved in this information community. We didn't, we don't have any evidence of pure haters, but what we observe again is that the two phenomena, so the polarization and hate speech are really interconnected and that we can observe a positive correlation between the toxicity and the length of the discussion. Then I am one minute late, so I will stop in just one minute just to say that I'm involved in other projects where, for example, with the Agicom, so the Italian Authority for Communications, I'm in a group of data science to study the phenomenon of disinformation. I also collaborated with the President of Council here in Italy on this topic. And we have now a really nice and interesting collaborating project, which is called Iris, with the Vaccine Confidence Project, the London School of Tropical Medicine in London, University of Cambridge, Sapienza University of Rome, City University of London and Harvard. And we are here studying infodemics and trying to understand how we can promote a more healthy, we can say, information ecosystem. I will stop here and leave some time for questions. Thank you very much.
 I will present very quickly some of the previous results that we had working with online data  and applying a data-driven approach to study social dynamics with online social media.  Then in the second part of this presentation, I will focus on some ongoing projects and some  recent results on polarization and also on the link between polarization and speech.  I would like to start from the digital information space. I work with mainly online data and with  data from social media platforms. With the event of the internet and later with social media,  we have lots of information that is being produced by users, so by us, when we use all  our devices in our digital lives, and also the fact that this information is a lot. We produce a  lot of information that, of course, can be precious if we want to run some kind of analysis,  but on the other side, the fact that we produce this information and that information is there  is also something that we, as users, have to process. The idea that we have lots of information  that is going around on the web and that can spread very easily and very quickly. This was a  great advantage and revolutionized the way in which we communicate with each other. The fact that  a piece of information can spread, so we have this first key point of this space. The fact  that information can spread very easily and very quickly and the fact that we have lots of information  to process. Also, the fact that this space, the digital information space, changed if we  compare this space to the traditional media system. The main difference is that while in the  traditional media system, the users, let's say, on the other side are passive, so they do not  have a direct, let's say, interaction with the information source. This is not happening on the  internet and, of course, it's not happening on social media, where the users play an active role  in not only accessing content and selecting the disinformation, but also in producing  their own information. This is another key point because we also observed the emergence of a more  heterogeneous system of new sources. For sure, this can also be an advantage. It can, for example,  foster a pluralism from a certain perspective. But what we also observe is that things are a bit  more complicated. What we observed in analyzing this, so what we do, just to quickly describe you  what we do is to collect and analyze data from online media platforms. We use methods from  computational social science, so we use methods from mathematics, statistics, computer science,  to analyze this data. But usually what we do is to answer questions that are  questions sometimes from the social sciences. It's a different approach. I would say a  complementary approach. It cannot totally substitute the traditional social sciences approach, but we  can have sometimes a broader overview of what's happening. Analyzing this data, what we have seen  is that the online digital system is characterized by strong polarization. We did lots of studies  on different topics from climate change to vaccines, to also science and pseudoscience,  and in all cases where the controversial issue is debated, the users polarized. You have users  that just got informed on a certain part, certain methods, a category of new sources,  and users on the other side of the spectrum. Although we can say that we could have access  to lots of information and information sources, what we observe is that in the end, the users tend  to confine their attention to well segregated groups of like-minded individuals. What I am  referring to are the now very famous eco chambers. The fact that we can observe the emergence of  this, we call them clusters in computer science. They are groups of individuals where you have  lots of internal connections. It's a very dense region in the network, but the connections between  the other groups with the other groups are not so many. The fact that you choose to interact  with a certain kind of information means that you are confining your attention to that narrative,  to that particular position, for example, on the topic, and you are not interacting and so having,  for example, a debate with the other side, with the other position. This can be, let's say,  a problem in the sense that it can reduce pluralism, for example, because what we also observed  is not only that the debate is very polarized and that these groups of like-minded individuals  emerge, but also we observed the selective exposure. The fact that even on more general,  so if we analyze all the information, for example, we had this study on more than 360 million users  on English sources on a global scale on Facebook. Also in that case, we found these communities,  these eco chambers were emerging. What we also observed was that users were,  so the number of new sources we interact with decreases with the increase of our activity on  the social media platform or with the increase of the time that our range would say on the platform.  So this means that, although potentially, we have access to this great amount of information,  what's happening in the end is that we are really confining our attention and so we are  selecting information from a very small set of new sources. These dynamics are, of course,  valid for information in general, but if we just think of misinformation as a particular kind of  information with specific characteristics, so for example, that it's not reliable,  you can understand that the mechanisms that are behind are the same. So in this direction,  it's very important to understand how these dynamics can also influence how misinformation,  for example, spreads and can spread and what are the factors that are behind this spreading.  What we also observed, for example, we studied the dynamics and the emotional dynamics of the  users involved in these communities and what we have seen is that, for example, the negative  emotional attitude of users increases with their polarization, but especially that the debate  degenerates when you have users from one community to interact and discuss with users  from the opposing community. So this is also showing us that we have this very polarized  environment, but what we also have is that users from different eco chambers are not only very  rarely interacting, but they're also, when they interact, most of the times this interaction is  not a civil discussion, a civil debate, but it's a debate that degenerates over time. So this  again can reduce the possibility of dialogue and debate, for example, in a democracy.  So what are the also the implication of this kind of system is that what we also did was to study how,  for example, the debunking content that was created on Facebook to debunk false claims and  hoaxes, how this content was received by, for example, pseudo scientific communities.  And what we observed was that this content, because of the eco chambers structure,  was not really reaching those communities. So we observed that the main consumers of this kind  of content are users from the scientific eco chambers, so users that we would expect not,  you know, needing this kind of corrections. And so, and this is the first point. So the fact  that having this really polarized environment and this structure in eco chambers can also limit  sometimes the contrasting approaches that we can implement. On the other side, what we also observed  was that in the cases where debunking was able to reach the target, let's say, in those cases,  we observed what we can define a backfire effect because these users from the pseudo scientific  community reinforced their attachment and their involvement in the pseudo scientific community  after being in contact with the debunking correction. So it was not from our data what we were  observing. So this kind of approach on social media was not really, was a bit ineffective.  And what I would like to say is that if you think about what I was saying before,  when we have a system like this where information gets spread very easily and very  and very fastly, the idea of debunking every single piece of content that is not  reliable is also feasible from a human point of view. And so what we tried to do was to try to  understand how, so what kind of approaches, what we can do to try to contrast misinformation from  a different point of view. So not acting when misinformation is out there, but try to do  something before. And what I would like to present today, so I will first talk about a paper work  that we did for the early warning, I will explain in a minute what I mean. And then I'll represent  some of the most recent projects I've been working on. So giving you some of, it's an idea of how we  can work with this kind of data. So what we observed looking at the data was that when we have a  certain topic appearing on official news, we have a sort of 24 hours window in which this topic can  appear on fake news. So the idea is that we don't have much time because it's only 24 hour window,  but we have some time maybe to make an intervention. And the idea of the intervention was that,  so if I'm able, let's say to predict what kind of topics, what kind of entities will become  subject, are likely, let's say to become subject of fake news in the short term, what I can do  is that I can try to adjust the communication and on these topics. Because what we observed  in all our studies was that we have this very strong link between misinformation and polarisation.  Lots of the times misinformation, especially disinformation, so when it's created a doc,  it's a kind of information that leverages the polarisation dynamics. And so what we did in  this study was to understand if looking, not at the content, so not at the, not trying to, you know,  define if the content is fake news or not, but try to understand how users are interacting  with content. If this information give us some hints to understand what's happening in the short,  in the short time, so in the future. So what we did was to, we developed a framework for  raising an early warning of potential misinformation targets. So the idea of this framework  was to being able to predict what topics are likely to be subject of misinformation in the  short term. So the idea, in the image that we have a system like this, for example, in press rooms,  what the journalist on the other side knows is that maybe what he can do is that adjust the  communication strategy on that particular topic so that he can try to reduce polarisation around  that topic and limit the spreading or the potential spreading on misinformation on that topic. So what  we did in this, what it is, what we did in this work is pretty technical in the sense that we had  the classifier, we developed a classifier to being able to raise this warning. But what I want just  to highlight here is the fact that this classifier is just using information from how users are  interacting with content, not the content itself. So looking at, for example, the sentiment of the  users involved in the discussion, the distance between how the, for example, the content is  presented, how the content is perceived, or how the content is perceived between, so in different  communities. So all these kinds of features were useful to produce a classifier that was able to  make this kind of distinction. So now imagine that we have a system like this. Now the question  is what we can do with this kind of system because, okay, I know that I have some topics, for example,  that can become, can become object of fake news, okay. What's the communication strategy that I  should adopt to try to understand, so to try to reduce that polarization because that was the aim  of the, so with the idea of I reduce the polarization and the discussion on the topic is less  polarized, maybe more civil. And so what happens is that the topic is not, is not,  is not something that can be of interest for the misinformation content. So we did the first,  let's say, experiment of this, working with the Lamos School of Economics and Corriere la Siera.  Corriere la Siera is the main newspaper that we have in Italy. The idea of this project was to try  to understand how the way in which you communicate the news impacted the way in which the users  shape the debate around that particular, that particular news. We analyzed in this project  data from Facebook and from Twitter and we monitored the reaction of readers to fact-based  journalistic outputs. In this project, which was the issue of migration, this was, so this is a  project from, that started in 2017. The migration issue was one of the hot topics in Italy, Italy's  agenda. And it was a very interesting debate, very polarized debate in our country. So it was a good  example to understand how we could try to, to shape the debate, to reduce the polarization around  the debate. What was interesting about the migration was also that we didn't have really an  emergency in Italy about migration. So the issue was stopping the newspaper's agenda, but we were  observing a strong decrease in your rivals in Italy. So, but the topic was very present also on  social media. And so it was also interesting to understand this kind of dynamics. So people were  really interested in a topic that it was not an emergency. And what was also, of course, it was  a topic that was greatly discussed at political level. And what, for example, we observed, and this  was very interesting, what you are seeing here on this graph, what we can define at the eco chambers.  If you look on the left side of this graph, you have on the circle, on the ring, what you have  are all the new sources that we have in Italy. And they have a different color according to the  community that belong to. So, and there is an edge, so a connection between two different new sources  if they have at least one user in common. So, using this information, we were able to  extract these three main communities. And these three main communities can be referred to the main,  to we can, the light blue one is what we call the mainstream media. Then we had the right in  the right in orange and in blue gray, we have the five star movement, let's say the populism,  the populist parties in Italy. What's very interesting is that usually in our works,  these kind of eco chambers are pretty stable over time. So they do not really change, okay,  if you observe what's happening over time, what we observed here was that in June 2018,  we had a need for the formation of a coalition government between the right and the five star  movement. And what we observed is that almost instantaneously, these two groups merged in one  group. So what you do, what you see in the following months is that from June to August, what we had  was the mainstream community and the orange is now representing the right, so the merging  between the right and the five star movement. So this is also explaining the fact that these  communities really emerge because of the, you know, selection of some information from users. So the  fact that now those new sources were together in the coalition government. And so the narrative  about migration was pretty similar between the two parties reflected in the way in which  also the news were communicating and the users were interacting with this information.  But what we did, so this was interesting to analyze the dynamics, but what we did was really  to perform an experiment where the journalist had to produce some stories on the topic of migration  and they shared these stories on Facebook. And the journalists, of course, were completely free  to choose the content of the news based on what was happening. But what we asked the journalist to  do was to try to use different content types and techniques when they were communicating the news.  So for example, to use infographics or plain text or to use to write an editorial or news reports  or human interest stories, so stories where they were telling the story of the people involved  in the news. And so for example, why they arrived in Italy, what they had to deal with before  coming here and so on. And we trapped all the data from the Facebook page in terms of engagement.  We also measured the sentiment of the comments of the users. And we had a group of human annotators  to annotate more than 26,000 comments, to assign three labels. The toxic label, so to understand if  the comment was aggressive or something that would let the user abandon a conversation.  And we used this label as a proxy for the civil discourse, mistrust of Corriere, so as a proxy  of trust of the news source. If for example, in the comment, we could have a direct attack  to the credibility of the source of information and the position on migration. So if the user was  in favor or against the migration phenomenon. And we used all this data to make some analysis  and to understand what was working better than other source techniques or types to smooth  polarization and facilitate a civil debate among the readers. And what we observed was that,  for example, all multimedia pieces and especially pieces that had videos inside were pretty,  so were very, received very strong and very supportive engagement with a higher number of likes  and shares and were very rarely criticized. We think that this can also depend on the fact that  having this kind of content, so for example pictures or videos in some sense makes more  difficult for users to go against the source. But on the other side, where all the content types  where the journalists were pushing facts to the forefront, so like for example infographics,  in this case, we had lots more debate and discussion and lots of pushback. So some sort of  backfire aspect that we can also see here and the high criticism of the media. This was something  that we were expecting. So the fact that having this kind of approach of a explicit direct  sometimes correction against a very sometimes emotional topic like the position on migration  couldn't work in the end. And for example, for the techniques, all the news reports that were  more, let's say straightforward, impartial, emotional, where the type of news that elicited  the most trust, but the human interest stories got a very strong pushback. This was not something  that the journalists were expecting because the idea of these stories, as I said before, was really  to sometimes also have a sort of more empathic approach. So to try to let people understand  what was really happening to these people and what were the reasons why these people were  coming to Italy. But on the other side, it can be that these kind of stories were felt as something,  you know, the user felt manipulated into taking a certain side because of this more emotional  news from the journalists. And in this case, we observed lots of toxic comments, lots of criticism,  lots of anti-migration comments. And in general, every time that the journalist provided a very  strong opinion of, for example, a policy proposal, like in the case of constructive journalism,  where they were proposing a solution, for example, to the problem, there was a lot of criticism of  the source. So the career was not trust by the commenters and lots of anti-migration comments.  So in general, using this data-driven approach, so like the infographics or giving, you know,  numbers and data to support the facts, was not really happening to reduce the polarization  of the debate. So of course, what I want to say is that these were interesting insights.  Of course, they're related to a specific topic. They're also related to our context in Italy.  And of course, there can also be an influence or from the same way in which the algorithms work,  for example, in trying to push some kind of content, like, for example, the multimedia content  over other kinds of content. But this was interesting also to understand that the way in which you do  your communication, so the way in which you decide to communicate the same piece of news,  can really change, can really shape how the debate, so the debate on the other side.  And so it can also be a way to try to think on how we can contrast, for example, misinformation  and polarization, not directly, but changing the approach on the communication side. And  speaking on the communication, from the communication side, what I want to mention very  quickly is that we also had this European project that was a two-year European project that  finished last July, where we analyzed how science is communicating in Europe. We did this for  seven countries and for the Twitter, Facebook and YouTube, so three different platforms,  and also for three controversial topics. And the results are interesting because they also show  differences in how the science communication is communicated, not only in the three platforms,  but also across the different countries. And so the fact that, again, when we analyze, for example,  social media data, it's also very important to take into account the fact that also how the people  and interact with information is also sometimes changing from country to country,  where we can, when users display different presences in how they interact with content.  If you are interested in this, there is the comprehensive report of this work on available,  I have the link here in the slide and on the website of the project.  What we did in the project was also to develop some quality indicators for science communication.  So what are the aspects that you have to take into account to improve and to ensure that  your communication is a quality communication? But what we also did for social media was that we  tested a series of recommendations for a period of five months, and then we developed this  recommendation for communicating science on social media. This was a pretty interesting,  again, experiment because the practices and the recommendations that we developed are data-driven  recommendations because we used the data that I was mentioning before to develop these recommendations,  together with the quality indicators that were developed through co-design activities and  qualitative research. And we put together all this information to develop these recommendations  that before being published, were also tested from communication practitioners on social media.  And again, all this information and these products are available on the website.  So now in the last, the other thing that I want to mention is that we are working also  on the infodemic and especially on COVID-19. So for example, we had this work where we  analyzed the COVID-19 infodemic on these five platforms and we observed that all these platforms  displayed the some, so displayed the really values that can be, so we measured the infodemic in a way  similar to how you can measure the epidemic. So we computed the R0 for all these platforms and  we observed that the, indeed, these values were hypercritical for all the platforms last year  at the beginning of the pandemic. And interesting, what we observed here was also that the way in which  misinformation was spreading in these platforms was similar to how reliable information was spreading.  So we didn't observe different patterns in how misinformation was spreading around.  What we observed was that the ratio of misinformation was higher in platforms where,  so not in mainstream platforms, but for example in Gabbo Reddit that are platforms where we have  less, let's say, moderation of content. And the last thing that I want to mention, the last project  in these 10 minutes I guess I have, is this project on 8-Speech is again a European project which has  several goals. So on one side, the detection of 8-Speech in different languages and the  identification of triggers and then the recommendation of counter narratives and the proposal of policies  to the European communication regulators. We now have, I will now show the first results  where we used the classifier for detecting 8-Speech and we did this study trying to understand if  there is a link between misinformation and 8-Speech. So the idea is that of course the detection of  this kind of, so the automatic detection of this kind of content is complicated and also the contrast  is complicated. For example, starting from the same definition, all the platforms have different  policies and have different definitions, for example, of what is considered hate speech or what's  speech that is not allowed by the platform. So the idea, so what we did in this work was to use  a definition of 8-Speech that refers to the whole spectrum of language that can be used in online  debates. So our definition includes normal, so from normal to, for example, acceptable,  inappropriate or violent speech and violent speech really cover all the forms that spread inside  and promote hate and but we also considered, for example, inappropriate or offensive speech.  So these kind of speeches are not illegal but they can deteriorate the public debate and they  can post radicalization. And we used this data to, so to first to develop the classifier. So we  used in this work data from the YouTube platform and the topic in Italian, so Italian from Italian  data. The topic was the COVID-19 and we have all this data and we also have channels and videos  from that where we can consider reliable and questionable. And what we did was to develop  this classification model. So how can you develop this? You take a sample of your comments and you  we hired a group of human annotators to annotate these comments across the different  labels, so appropriate, inappropriate, offensive, violent. And then when you have this data, you use  this data to train the classification, so to train the algorithm so that the algorithm is able to  understand, so to evaluate if a certain, so to learn from the human annotations how it can  identify comments in the different in the different labels. And of course the model then is evaluated.  And when this process is concluded, you have a classifier that works with a certain accuracy.  What you do is also that you compare the, so how good your model is against how good the  annotators were. So in our case, our model is pretty close to what the annotators, to the agreement  between and among the annotators in our, in our class annotation task. And so we can say that we  have a very high quality model. Once you have the model, what you do is that you apply the model  to the whole data set of comments because of course it's not possible to manually annotate  millions of comments. So the idea of using these classificators is that you try to use this,  you use the model to classify the rest of the, the rest of the comments.  So what we observe is that there is the proportion of the hate speech labels, the forward speech  labels in the whole data set are similar for questionable and for reliable channels. So we  don't observe any particular differences. And we also measured the comment delay. So the time  that elapsed between the posting time of a video, so when a video is posted and the comment that,  the first comment that arrived, arrives to the video in hours. And what we see is that there is,  so if we look at the different kinds of comments, there is not great difference, but on questionable  channels, toxic comments appear first and faster than appropriate ones. And following the decreasing  levels of toxicity. So when we are on questionable channels, so channels that we can, where we can  have fake news or all, you know, that kind of unreliable information, the toxic comments,  so comments that we can define offensive, violent or inappropriate, arrive first and faster than  the, than the others. What we also observe, and this is pretty interesting, was that we were not  able to observe the presence of what we called the pure haters. So the users that only use a  violent language, this was not the case in our data. So when you look at the data, what you see  is that we have users that are what we can say, you know, normal users that happen to use a violent,  so hate speech in some situations. And indeed, one of the, you know, the future work is devoted to  the understanding what kind of triggers, okay, you can have in the, in the, you can have there to try to  to foster this kind of behavior. So we have very few users in our data set that only use violent  comments, but their activity is pretty low. So they have less than five comments. The other thing  that we observed was that, so we computed the, let's say, polarization of these users. So if they  were using and getting videos from reliable channel or from questionable channels. And what we  observe is that even here we have two peaks, okay, in correspondence of these extreme values of  linings. So what we, what we can say is that polarize, more polarized users tend to use  a higher proportions of non-appropriate violent toxic comments. But interestingly, the users skewed  towards reliable channels, post on average, a higher proportion of non-appropriate comments,  then users skewed towards questionable channels. And what's also coming from this data is that  these users mainly use inappropriate comments in their opponents community. So again, if you  remember what I was saying at the beginning about the, how these, the, you know, interaction dynamics  take form and the fact that the debate is degenerating, we are confirming what we were observing  and the level of sentiment, even looking at hate speech. And what we observe is also in this case  that the toxicity level, so the average of the toxicity values correlates possibly  with the length of the discussion. So the more the discussion, so when the discussion increases  in terms of comments and in terms of time, the toxicity of this discussion increases.  So we have a more toxic debate when the discussion gets longer. So, yeah, this is just a summary of  the fact that we didn't find any evidence of relationship between the use of a toxic, toxic  language and being involved in this information community. We didn't, we don't have any evidence  of pure haters, but what we observe again is that the two phenomena, so the polarization and hate  speech are really interconnected and that we can observe a positive correlation between the toxicity  and the length of the discussion. Then I am one minute late, so I will stop in just one minute  just to say that I'm involved in other projects where, for example, with the Agicom,  so the Italian Authority for Communications, I'm in a group of data science to study the  phenomenon of disinformation. I also collaborated with the President of Council here in Italy on  this topic. And we have now a really nice and interesting collaborating project,  which is called Iris, with the Vaccine Confidence Project, the London School of Tropical Medicine  in London, University of Cambridge, Sapienza University of Rome, City University of London  and Harvard. And we are here studying infodemics and trying to understand how we can promote a more  healthy, we can say, information ecosystem. I will stop here and leave some time for questions. Thank you very much.
Keynote_Zollo_0_107.png Keynote_Zollo_108_203.png Keynote_Zollo_204_359.png Keynote_Zollo_360_467.png Keynote_Zollo_468_599.png Keynote_Zollo_600_659.png Keynote_Zollo_660_731.png Keynote_Zollo_732_851.png Keynote_Zollo_852_983.png Keynote_Zollo_984_1103.png Keynote_Zollo_1104_1151.png Keynote_Zollo_1152_1223.png Keynote_Zollo_1224_1295.png Keynote_Zollo_1296_6875.png Keynote_Zollo_6876_8363.png Keynote_Zollo_8364_10331.png Keynote_Zollo_10332_13175.png Keynote_Zollo_13176_15107.png Keynote_Zollo_15108_15251.png Keynote_Zollo_15252_19391.png Keynote_Zollo_19392_20123.png Keynote_Zollo_20124_22811.png Keynote_Zollo_22812_26999.png Keynote_Zollo_27000_27119.png Keynote_Zollo_27120_27383.png Keynote_Zollo_27384_27743.png Keynote_Zollo_27744_27875.png Keynote_Zollo_27876_27995.png Keynote_Zollo_27996_28379.png Keynote_Zollo_28380_28499.png Keynote_Zollo_28500_29015.png Keynote_Zollo_29016_30011.png Keynote_Zollo_30012_34355.png Keynote_Zollo_34356_37799.png Keynote_Zollo_37800_40067.png Keynote_Zollo_40068_44855.png Keynote_Zollo_44856_46931.png Keynote_Zollo_46932_47639.png Keynote_Zollo_47640_48923.png Keynote_Zollo_48924_49151.png Keynote_Zollo_49152_51179.png Keynote_Zollo_51180_52367.png Keynote_Zollo_52368_53327.png Keynote_Zollo_53328_54383.png Keynote_Zollo_54384_55247.png Keynote_Zollo_55248_56687.png Keynote_Zollo_56688_58055.png Keynote_Zollo_58056_58127.png Keynote_Zollo_58128_58619.png Keynote_Zollo_58620_60179.png Keynote_Zollo_60180_61787.png Keynote_Zollo_61788_63815.png Keynote_Zollo_63816_64691.png Keynote_Zollo_64692_65423.png Keynote_Zollo_65424_67079.png Keynote_Zollo_67080_67127.png Keynote_Zollo_67128_67246.png
I will present very quickly some of the previous results that we had working with online data and applying a data-driven approach to study social dynamics with online social media. Then in the second part of this presentation, I will focus on some ongoing projects and some recent results on polarization and also on the link between polarization and speech. I would like to start from the digital information space. I work with mainly online data and with data from social media platforms. With the event of the internet and later with social media, we have lots of information that is being produced by users, so by us, when we use all our devices in our digital lives, and also the fact that this information is a lot. We produce a lot of information that, of course, can be precious if we want to run some kind of analysis, but on the other side, the fact that we produce this information and that information is there is also something that we, as users, have to process. The idea that we have lots of information that is going around on the web and that can spread very easily and very quickly. This was a great advantage and revolutionized the way in which we communicate with each other. The fact that a piece of information can spread, so we have this first key point of this space. The fact that information can spread very easily and very quickly and the fact that we have lots of information to process. Also, the fact that this space, the digital information space, changed if we compare this space to the traditional media system. The main difference is that while in the traditional media system, the users, let's say, on the other side are passive, so they do not have a direct, let's say, interaction with the information source. This is not happening on the internet and, of course, it's not happening on social media, where the users play an active role in not only accessing content and selecting the disinformation, but also in producing their own information. This is another key point because we also observed the emergence of a more heterogeneous system of new sources. For sure, this can also be an advantage. It can, for example, foster a pluralism from a certain perspective. But what we also observe is that things are a bit more complicated. What we observed in analyzing this, so what we do, just to quickly describe you what we do is to collect and analyze data from online media platforms. We use methods from computational social science, so we use methods from mathematics, statistics, computer science, to analyze this data. But usually what we do is to answer questions that are questions sometimes from the social sciences. It's a different approach. I would say a complementary approach. It cannot totally substitute the traditional social sciences approach, but we can have sometimes a broader overview of what's happening. Analyzing this data, what we have seen is that the online digital system is characterized by strong polarization. We did lots of studies on different topics from climate change to vaccines, to also science and pseudoscience, and in all cases where the controversial issue is debated, the users polarized. You have users that just got informed on a certain part, certain methods, a category of new sources, and users on the other side of the spectrum. Although we can say that we could have access to lots of information and information sources, what we observe is that in the end, the users tend to confine their attention to well segregated groups of like-minded individuals. What I am referring to are the now very famous eco chambers. The fact that we can observe the emergence of this, we call them clusters in computer science. They are groups of individuals where you have lots of internal connections. It's a very dense region in the network, but the connections between the other groups with the other groups are not so many. The fact that you choose to interact with a certain kind of information means that you are confining your attention to that narrative, to that particular position, for example, on the topic, and you are not interacting and so having, for example, a debate with the other side, with the other position. This can be, let's say, a problem in the sense that it can reduce pluralism, for example, because what we also observed is not only that the debate is very polarized and that these groups of like-minded individuals emerge, but also we observed the selective exposure. The fact that even on more general, so if we analyze all the information, for example, we had this study on more than 360 million users on English sources on a global scale on Facebook. Also in that case, we found these communities, these eco chambers were emerging. What we also observed was that users were, so the number of new sources we interact with decreases with the increase of our activity on the social media platform or with the increase of the time that our range would say on the platform. So this means that, although potentially, we have access to this great amount of information, what's happening in the end is that we are really confining our attention and so we are selecting information from a very small set of new sources. These dynamics are, of course, valid for information in general, but if we just think of misinformation as a particular kind of information with specific characteristics, so for example, that it's not reliable, you can understand that the mechanisms that are behind are the same. So in this direction, it's very important to understand how these dynamics can also influence how misinformation, for example, spreads and can spread and what are the factors that are behind this spreading. What we also observed, for example, we studied the dynamics and the emotional dynamics of the users involved in these communities and what we have seen is that, for example, the negative emotional attitude of users increases with their polarization, but especially that the debate degenerates when you have users from one community to interact and discuss with users from the opposing community. So this is also showing us that we have this very polarized environment, but what we also have is that users from different eco chambers are not only very rarely interacting, but they're also, when they interact, most of the times this interaction is not a civil discussion, a civil debate, but it's a debate that degenerates over time. So this again can reduce the possibility of dialogue and debate, for example, in a democracy. So what are the also the implication of this kind of system is that what we also did was to study how, for example, the debunking content that was created on Facebook to debunk false claims and hoaxes, how this content was received by, for example, pseudo scientific communities. And what we observed was that this content, because of the eco chambers structure, was not really reaching those communities. So we observed that the main consumers of this kind of content are users from the scientific eco chambers, so users that we would expect not, you know, needing this kind of corrections. And so, and this is the first point. So the fact that having this really polarized environment and this structure in eco chambers can also limit sometimes the contrasting approaches that we can implement. On the other side, what we also observed was that in the cases where debunking was able to reach the target, let's say, in those cases, we observed what we can define a backfire effect because these users from the pseudo scientific community reinforced their attachment and their involvement in the pseudo scientific community after being in contact with the debunking correction. So it was not from our data what we were observing. So this kind of approach on social media was not really, was a bit ineffective. And what I would like to say is that if you think about what I was saying before, when we have a system like this where information gets spread very easily and very and very fastly, the idea of debunking every single piece of content that is not reliable is also feasible from a human point of view. And so what we tried to do was to try to understand how, so what kind of approaches, what we can do to try to contrast misinformation from a different point of view. So not acting when misinformation is out there, but try to do something before. And what I would like to present today, so I will first talk about a paper work that we did for the early warning, I will explain in a minute what I mean. And then I'll represent some of the most recent projects I've been working on. So giving you some of, it's an idea of how we can work with this kind of data. So what we observed looking at the data was that when we have a certain topic appearing on official news, we have a sort of 24 hours window in which this topic can appear on fake news. So the idea is that we don't have much time because it's only 24 hour window, but we have some time maybe to make an intervention. And the idea of the intervention was that, so if I'm able, let's say to predict what kind of topics, what kind of entities will become subject, are likely, let's say to become subject of fake news in the short term, what I can do is that I can try to adjust the communication and on these topics. Because what we observed in all our studies was that we have this very strong link between misinformation and polarisation. Lots of the times misinformation, especially disinformation, so when it's created a doc, it's a kind of information that leverages the polarisation dynamics. And so what we did in this study was to understand if looking, not at the content, so not at the, not trying to, you know, define if the content is fake news or not, but try to understand how users are interacting with content. If this information give us some hints to understand what's happening in the short, in the short time, so in the future. So what we did was to, we developed a framework for raising an early warning of potential misinformation targets. So the idea of this framework was to being able to predict what topics are likely to be subject of misinformation in the short term. So the idea, in the image that we have a system like this, for example, in press rooms, what the journalist on the other side knows is that maybe what he can do is that adjust the communication strategy on that particular topic so that he can try to reduce polarisation around that topic and limit the spreading or the potential spreading on misinformation on that topic. So what we did in this, what it is, what we did in this work is pretty technical in the sense that we had the classifier, we developed a classifier to being able to raise this warning. But what I want just to highlight here is the fact that this classifier is just using information from how users are interacting with content, not the content itself. So looking at, for example, the sentiment of the users involved in the discussion, the distance between how the, for example, the content is presented, how the content is perceived, or how the content is perceived between, so in different communities. So all these kinds of features were useful to produce a classifier that was able to make this kind of distinction. So now imagine that we have a system like this. Now the question is what we can do with this kind of system because, okay, I know that I have some topics, for example, that can become, can become object of fake news, okay. What's the communication strategy that I should adopt to try to understand, so to try to reduce that polarization because that was the aim of the, so with the idea of I reduce the polarization and the discussion on the topic is less polarized, maybe more civil. And so what happens is that the topic is not, is not, is not something that can be of interest for the misinformation content. So we did the first, let's say, experiment of this, working with the Lamos School of Economics and Corriere la Siera. Corriere la Siera is the main newspaper that we have in Italy. The idea of this project was to try to understand how the way in which you communicate the news impacted the way in which the users shape the debate around that particular, that particular news. We analyzed in this project data from Facebook and from Twitter and we monitored the reaction of readers to fact-based journalistic outputs. In this project, which was the issue of migration, this was, so this is a project from, that started in 2017. The migration issue was one of the hot topics in Italy, Italy's agenda. And it was a very interesting debate, very polarized debate in our country. So it was a good example to understand how we could try to, to shape the debate, to reduce the polarization around the debate. What was interesting about the migration was also that we didn't have really an emergency in Italy about migration. So the issue was stopping the newspaper's agenda, but we were observing a strong decrease in your rivals in Italy. So, but the topic was very present also on social media. And so it was also interesting to understand this kind of dynamics. So people were really interested in a topic that it was not an emergency. And what was also, of course, it was a topic that was greatly discussed at political level. And what, for example, we observed, and this was very interesting, what you are seeing here on this graph, what we can define at the eco chambers. If you look on the left side of this graph, you have on the circle, on the ring, what you have are all the new sources that we have in Italy. And they have a different color according to the community that belong to. So, and there is an edge, so a connection between two different new sources if they have at least one user in common. So, using this information, we were able to extract these three main communities. And these three main communities can be referred to the main, to we can, the light blue one is what we call the mainstream media. Then we had the right in the right in orange and in blue gray, we have the five star movement, let's say the populism, the populist parties in Italy. What's very interesting is that usually in our works, these kind of eco chambers are pretty stable over time. So they do not really change, okay, if you observe what's happening over time, what we observed here was that in June 2018, we had a need for the formation of a coalition government between the right and the five star movement. And what we observed is that almost instantaneously, these two groups merged in one group. So what you do, what you see in the following months is that from June to August, what we had was the mainstream community and the orange is now representing the right, so the merging between the right and the five star movement. So this is also explaining the fact that these communities really emerge because of the, you know, selection of some information from users. So the fact that now those new sources were together in the coalition government. And so the narrative about migration was pretty similar between the two parties reflected in the way in which also the news were communicating and the users were interacting with this information. But what we did, so this was interesting to analyze the dynamics, but what we did was really to perform an experiment where the journalist had to produce some stories on the topic of migration and they shared these stories on Facebook. And the journalists, of course, were completely free to choose the content of the news based on what was happening. But what we asked the journalist to do was to try to use different content types and techniques when they were communicating the news. So for example, to use infographics or plain text or to use to write an editorial or news reports or human interest stories, so stories where they were telling the story of the people involved in the news. And so for example, why they arrived in Italy, what they had to deal with before coming here and so on. And we trapped all the data from the Facebook page in terms of engagement. We also measured the sentiment of the comments of the users. And we had a group of human annotators to annotate more than 26,000 comments, to assign three labels. The toxic label, so to understand if the comment was aggressive or something that would let the user abandon a conversation. And we used this label as a proxy for the civil discourse, mistrust of Corriere, so as a proxy of trust of the news source. If for example, in the comment, we could have a direct attack to the credibility of the source of information and the position on migration. So if the user was in favor or against the migration phenomenon. And we used all this data to make some analysis and to understand what was working better than other source techniques or types to smooth polarization and facilitate a civil debate among the readers. And what we observed was that, for example, all multimedia pieces and especially pieces that had videos inside were pretty, so were very, received very strong and very supportive engagement with a higher number of likes and shares and were very rarely criticized. We think that this can also depend on the fact that having this kind of content, so for example pictures or videos in some sense makes more difficult for users to go against the source. But on the other side, where all the content types where the journalists were pushing facts to the forefront, so like for example infographics, in this case, we had lots more debate and discussion and lots of pushback. So some sort of backfire aspect that we can also see here and the high criticism of the media. This was something that we were expecting. So the fact that having this kind of approach of a explicit direct sometimes correction against a very sometimes emotional topic like the position on migration couldn't work in the end. And for example, for the techniques, all the news reports that were more, let's say straightforward, impartial, emotional, where the type of news that elicited the most trust, but the human interest stories got a very strong pushback. This was not something that the journalists were expecting because the idea of these stories, as I said before, was really to sometimes also have a sort of more empathic approach. So to try to let people understand what was really happening to these people and what were the reasons why these people were coming to Italy. But on the other side, it can be that these kind of stories were felt as something, you know, the user felt manipulated into taking a certain side because of this more emotional news from the journalists. And in this case, we observed lots of toxic comments, lots of criticism, lots of anti-migration comments. And in general, every time that the journalist provided a very strong opinion of, for example, a policy proposal, like in the case of constructive journalism, where they were proposing a solution, for example, to the problem, there was a lot of criticism of the source. So the career was not trust by the commenters and lots of anti-migration comments. So in general, using this data-driven approach, so like the infographics or giving, you know, numbers and data to support the facts, was not really happening to reduce the polarization of the debate. So of course, what I want to say is that these were interesting insights. Of course, they're related to a specific topic. They're also related to our context in Italy. And of course, there can also be an influence or from the same way in which the algorithms work, for example, in trying to push some kind of content, like, for example, the multimedia content over other kinds of content. But this was interesting also to understand that the way in which you do your communication, so the way in which you decide to communicate the same piece of news, can really change, can really shape how the debate, so the debate on the other side. And so it can also be a way to try to think on how we can contrast, for example, misinformation and polarization, not directly, but changing the approach on the communication side. And speaking on the communication, from the communication side, what I want to mention very quickly is that we also had this European project that was a two-year European project that finished last July, where we analyzed how science is communicating in Europe. We did this for seven countries and for the Twitter, Facebook and YouTube, so three different platforms, and also for three controversial topics. And the results are interesting because they also show differences in how the science communication is communicated, not only in the three platforms, but also across the different countries. And so the fact that, again, when we analyze, for example, social media data, it's also very important to take into account the fact that also how the people and interact with information is also sometimes changing from country to country, where we can, when users display different presences in how they interact with content. If you are interested in this, there is the comprehensive report of this work on available, I have the link here in the slide and on the website of the project. What we did in the project was also to develop some quality indicators for science communication. So what are the aspects that you have to take into account to improve and to ensure that your communication is a quality communication? But what we also did for social media was that we tested a series of recommendations for a period of five months, and then we developed this recommendation for communicating science on social media. This was a pretty interesting, again, experiment because the practices and the recommendations that we developed are data-driven recommendations because we used the data that I was mentioning before to develop these recommendations, together with the quality indicators that were developed through co-design activities and qualitative research. And we put together all this information to develop these recommendations that before being published, were also tested from communication practitioners on social media. And again, all this information and these products are available on the website. So now in the last, the other thing that I want to mention is that we are working also on the infodemic and especially on COVID-19. So for example, we had this work where we analyzed the COVID-19 infodemic on these five platforms and we observed that all these platforms displayed the some, so displayed the really values that can be, so we measured the infodemic in a way similar to how you can measure the epidemic. So we computed the R0 for all these platforms and we observed that the, indeed, these values were hypercritical for all the platforms last year at the beginning of the pandemic. And interesting, what we observed here was also that the way in which misinformation was spreading in these platforms was similar to how reliable information was spreading. So we didn't observe different patterns in how misinformation was spreading around. What we observed was that the ratio of misinformation was higher in platforms where, so not in mainstream platforms, but for example in Gabbo Reddit that are platforms where we have less, let's say, moderation of content. And the last thing that I want to mention, the last project in these 10 minutes I guess I have, is this project on 8-Speech is again a European project which has several goals. So on one side, the detection of 8-Speech in different languages and the identification of triggers and then the recommendation of counter narratives and the proposal of policies to the European communication regulators. We now have, I will now show the first results where we used the classifier for detecting 8-Speech and we did this study trying to understand if there is a link between misinformation and 8-Speech. So the idea is that of course the detection of this kind of, so the automatic detection of this kind of content is complicated and also the contrast is complicated. For example, starting from the same definition, all the platforms have different policies and have different definitions, for example, of what is considered hate speech or what's speech that is not allowed by the platform. So the idea, so what we did in this work was to use a definition of 8-Speech that refers to the whole spectrum of language that can be used in online debates. So our definition includes normal, so from normal to, for example, acceptable, inappropriate or violent speech and violent speech really cover all the forms that spread inside and promote hate and but we also considered, for example, inappropriate or offensive speech. So these kind of speeches are not illegal but they can deteriorate the public debate and they can post radicalization. And we used this data to, so to first to develop the classifier. So we used in this work data from the YouTube platform and the topic in Italian, so Italian from Italian data. The topic was the COVID-19 and we have all this data and we also have channels and videos from that where we can consider reliable and questionable. And what we did was to develop this classification model. So how can you develop this? You take a sample of your comments and you we hired a group of human annotators to annotate these comments across the different labels, so appropriate, inappropriate, offensive, violent. And then when you have this data, you use this data to train the classification, so to train the algorithm so that the algorithm is able to understand, so to evaluate if a certain, so to learn from the human annotations how it can identify comments in the different in the different labels. And of course the model then is evaluated. And when this process is concluded, you have a classifier that works with a certain accuracy. What you do is also that you compare the, so how good your model is against how good the annotators were. So in our case, our model is pretty close to what the annotators, to the agreement between and among the annotators in our, in our class annotation task. And so we can say that we have a very high quality model. Once you have the model, what you do is that you apply the model to the whole data set of comments because of course it's not possible to manually annotate millions of comments. So the idea of using these classificators is that you try to use this, you use the model to classify the rest of the, the rest of the comments. So what we observe is that there is the proportion of the hate speech labels, the forward speech labels in the whole data set are similar for questionable and for reliable channels. So we don't observe any particular differences. And we also measured the comment delay. So the time that elapsed between the posting time of a video, so when a video is posted and the comment that, the first comment that arrived, arrives to the video in hours. And what we see is that there is, so if we look at the different kinds of comments, there is not great difference, but on questionable channels, toxic comments appear first and faster than appropriate ones. And following the decreasing levels of toxicity. So when we are on questionable channels, so channels that we can, where we can have fake news or all, you know, that kind of unreliable information, the toxic comments, so comments that we can define offensive, violent or inappropriate, arrive first and faster than the, than the others. What we also observe, and this is pretty interesting, was that we were not able to observe the presence of what we called the pure haters. So the users that only use a violent language, this was not the case in our data. So when you look at the data, what you see is that we have users that are what we can say, you know, normal users that happen to use a violent, so hate speech in some situations. And indeed, one of the, you know, the future work is devoted to the understanding what kind of triggers, okay, you can have in the, in the, you can have there to try to to foster this kind of behavior. So we have very few users in our data set that only use violent comments, but their activity is pretty low. So they have less than five comments. The other thing that we observed was that, so we computed the, let's say, polarization of these users. So if they were using and getting videos from reliable channel or from questionable channels. And what we observe is that even here we have two peaks, okay, in correspondence of these extreme values of linings. So what we, what we can say is that polarize, more polarized users tend to use a higher proportions of non-appropriate violent toxic comments. But interestingly, the users skewed towards reliable channels, post on average, a higher proportion of non-appropriate comments, then users skewed towards questionable channels. And what's also coming from this data is that these users mainly use inappropriate comments in their opponents community. So again, if you remember what I was saying at the beginning about the, how these, the, you know, interaction dynamics take form and the fact that the debate is degenerating, we are confirming what we were observing and the level of sentiment, even looking at hate speech. And what we observe is also in this case that the toxicity level, so the average of the toxicity values correlates possibly with the length of the discussion. So the more the discussion, so when the discussion increases in terms of comments and in terms of time, the toxicity of this discussion increases. So we have a more toxic debate when the discussion gets longer. So, yeah, this is just a summary of the fact that we didn't find any evidence of relationship between the use of a toxic, toxic language and being involved in this information community. We didn't, we don't have any evidence of pure haters, but what we observe again is that the two phenomena, so the polarization and hate speech are really interconnected and that we can observe a positive correlation between the toxicity and the length of the discussion. Then I am one minute late, so I will stop in just one minute just to say that I'm involved in other projects where, for example, with the Agicom, so the Italian Authority for Communications, I'm in a group of data science to study the phenomenon of disinformation. I also collaborated with the President of Council here in Italy on this topic. And we have now a really nice and interesting collaborating project, which is called Iris, with the Vaccine Confidence Project, the London School of Tropical Medicine in London, University of Cambridge, Sapienza University of Rome, City University of London and Harvard. And we are here studying infodemics and trying to understand how we can promote a more healthy, we can say, information ecosystem. I will stop here and leave some time for questions. Thank you very much.
dynamics with online social media. Then in the space to the traditional media system. The main disinformation, but also in producing their own this can also be an advantage. It can, for overview of what's happening. Analyzing this new sources, and users on the other side of the access to lots of information and information position. This can be, let's say, a problem in all the information, for example, we had this the platform. So this means that, although misinformation, for example, spreads and can example, we studied the dynamics and the warning, I will explain in a minute what I mean. fact-based journalistic outputs. In this project, news were communicating and the users were all the content types where the journalists were recommendations because we used the data that I patterns in how misinformation was spreading not illegal but they can deteriorate the public COVID-19 and we have all this data and we also