Build a Twitter topic extractor
Please read the topic categorydescriptionto understand what this is all aboutDescriptionTwitter classifies trending tweets according to predefined set of topics like “Data science”, “Hip hop”, “Sport” etc. However, their algorithm appears to often get confused by certain keywords in the tweet or the content of the image (seeherefor some funny examples).The goal of this project is to explore whether it’s possible to create a better topic extractor, or at least one that is more targeted for a smaller set of domains. There’s several ways to approach to task:Frame it as a multiclass classification problemFrame it as an unsupervised clustering problem, and combine this with techniques likeUMAPand/or HDBSCANModel(s)Since Tweets are short in length, picking one of thesentence-transformersmodels on the Hubis likely a good place to start.DatasetsThere are various Twitter datasets on the Hub and here’s a few examples to start with:tweet_evalsentiment140emotionChallengesIf you take the unsupervised learning approach, be warned that there’s no “correct” answer and you will have to experiment with the dimensionality reduction / cluster algorithms to get meaningful clusters.Desired project outcomesCreate a Streamlit or Gradio app onSpacesthat either can automatically classify a Tweet according to a topic, or visualises the 2D projection of the embeddings and colours them by topic.Don’t forget to push all your models and datasets to the Hub so others can build on them!Additional resourceshttps://towardsdatascience.com/how-exactly-umap-works-13e3040e1668https://huggingface.co/spaces/edugp/embedding-lenses(useful for UMAP inspiration)Discord channelTo chat and organise with other people interested in this project, head over to ourDiscordand:Follow the instructions on the#join-coursechannelJoin the#twitter-topic-extractorchannelJust make sure you comment here to indicate that you’ll be contributing to this project
2021-11-15T20:11:26Z
I’d like to work on this project!
2021-11-16T21:53:40Z
I’d like to join this project!
2021-11-17T00:27:02Z
I am interested as well.  What is your timeframe?
2021-11-17T01:27:37Z
I think that the project needs to be available for reviewal in November 24th.
2021-11-17T05:29:52Z
Hello Wilmer,Would you like to have a zoom chat tomorrow (after the class)?I am in Calif (-3 hours from you)Al
2021-11-19T02:53:00Z
Oh sorry didn’t see this, Yes, let’s catch up in discord
2023-03-07T13:43:00Z
Interested