- Heba Elfardy, Mona T. Diab. 2016. Addressing Annotation Complexity: The Case of Annotating Ideological Perspective in Egyptian Social Media. Abstract: Automatically detecting the stance of people toward political and ideological topics –namely their “Ideological Perspective”– from social media is a rapidly growing research area with a wide range of applications. Research in such a ﬁeld faces several challenges among which is the lack of annotated corpora and associated guidelines for collecting annotations. The problem is even more pronounced in situations where there is no clear taxonomy for the common community perspectives and ideologies. The challenges are exacerbated when the communities where we need to gather these annotations are in a state of turmoil causing subjectivity and intimidation to be factors in the annotation process. Accordingly, we present the process for creating a robust and succinct set of guidelines for annotating “Egyp-tian Ideological Perspectives”. We collect social media data discussing Egyptian politics and develop an iterative feedback annotation framework reﬁning the annotation task and associated guidelines at-tempting to circumvent both weaknesses. Our efforts lead to a signiﬁcant increase in inter-annotator agreement measures from 75.7% to 92% overall agreement.
