Learning from emojis
Please read the topic categorydescriptionto understand what this is all aboutDescriptionEmoticons are often used as a proxy for emotional content in social media posts or instant messaging chats. As a result, emojis are often used as a label to train text classifiers. The goal of this project is to create a Transformer-based implementation ofDeepEmoji, a research project from MIT that studied this task with LSTMs.Model(s)Any BERT-like model would be a good candidate for fine-tuning on an emoji dataset and you can get inspiration from models like these:cardiffnlp/twitter-roberta-base-emojiDatasetstweet_evalChallengesTo get better performance, you may want to perform domain adaptation by fine-tuning the language model on in-domain data. We recommend trying this approach only after building a baseline classifier.Desired project outcomesCreate a Streamlit or Gradio app onSpacesthat can predict the top 5 emojis associated with a piece of textDon’t forget to push all your models and datasets to the Hub so others can build on them!Additional resourceshttps://huggingface.co/spaces/ml6team/emoji_predictor(a Space for inspiration)Discord channelTo chat and organise with other people interested in this project, head over to ourDiscordand:Follow the instructions on the#join-coursechannelJoin thelearning-from-emojischannelJust make sure you comment here to indicate that you’ll be contributing to this project
2021-11-14T05:31:25Z
This project sounds interesting. I would like to work on something like this.
2021-11-14T10:51:56Z
Hey@saldera, I’ve added some details about the Discord channel associated with this project in case you and others wish to coordinate over there
2021-11-15T10:28:54Z
Hi,I would like to participate, but I want to add a slight twist to the idea. So basically I’m thinking about fine-tuning a multilingual model (or bilingual for that matter) on tweet_eval, and then performing sort of cross-lingual zero-shot classification on the other language(s) (it’s just Russian in my case). This works surprisingly well on MNLI which is, arguably, more difficult, so it’s worth a shot, I think.
2021-11-15T12:50:14Z
Hey@adorkin, this sounds like a really nice idea!
2021-11-16T09:49:20Z
I made a really primitivedemo. It seems to me that emojis in tweet eval are kind of same-ish, and, cosequently, predicted emojis are also similar for different texts. Any feedback?
2021-11-16T09:51:13Z
What I mean is that there’s essentially only one kind of sentiment for every text, which seems boring.
2021-11-16T10:06:35Z
Wow, congrats on tuning the model / building the Space so fast!I think you’re right that the model seems to only predict positive sentiments - what I suggest is looking at the distribution of emojis intweet_evaland perhaps consider filtering the dataset for a select number of emojis that capture various emotionsScreen Shot 2021-11-16 at 11.04.121528×359 27 KB
2021-11-16T11:15:10Z
It is an option, however I don’t think it will change the results dramatically because, again, all the emojis in this particular dataset are not really that different even if you do filter them a bit (it’s still going to remain positive, basically). For example, what exactly distinguishes several different kinds of heart emojis? What I mean is that keeping, let’s say, only a smiley, a heart and a tree won’t really make it more interesting, unfortunately. There’s not even a sad emojiMeanwhile, the DeepEmoji demo definitely offers a much larger range of emojis. I can’t figure out what data it was trained on though. It would be nice to use it.I was thinking about maybe doing sort of augmentation by means of replacing emotion tags with emojis in the same dataset in combination with the aforementioned filtering. It’s not anexactsolution, but it should be okay for a fun project.
2021-11-16T11:34:46Z
For reference,here’sthe full set of emojis in tweet eval.
2021-11-16T11:55:56Z
I am also working on this, helping Aleksey with the UI and stuff
2021-11-16T12:11:23Z
Thanks for digging into the label set - you’re totally right that this is pretty biased! I think your approach of mapping emoji tags to emojis could work with some historical twitter datasets (likethis) to collect more diverse labels.I saw that the DeepEmoji team released somebenchmarkdatasets, but haven’t checked their content as to their use
2021-11-19T11:45:38Z
Okay, so I tried diversifying the labels using the approach that I mentioned previously, and I also added class weights like you showed in your tutorial. The outputs are a bit more reasonable now, although there’s evidently still not enough non-positive emojis to fill five slots.image743×356 17.8 KBHere’s the demo made withakshay7’s help.
2021-11-19T13:45:18Z
Cool to hear that you could use the tricks from the tutorial! The Space is really nice and I think the model does a decent job at capturing the top 1-2 emojis
2021-11-19T13:58:37Z
I think the most interesting thing here is that it also works fairly well with a language it was never trained on. Apparently It’s even able to pick up on the usage of brackets (instead of fully-fledged emojis) to express emotions, which is not a thing on the English-speaking internet, as far as I’m aware. I added some examples from twitter datasets to demonstrate that.On a separate note, maybe it’s not entirely fair to expect the model to predict the same emojis that one would expect it to, because in many cases it’s not really possible to figure the exact emotion being expressed without extratextual cues that are partially substitued by emojis (and brackets, I guess) in the first place.
2021-11-19T20:02:37Z
@adorkin@akshay7This is really cool. Good job!Unfortunately I haven’t been able to participate due to exams.
2021-11-21T13:16:37Z
Hey@saldera, not an issue. Please try out the app though as I think its ready nowTwitterEmojis - a Hugging Face Space by learningfromemojis