Sentiment Analysis keywords
My objective is to understand the keywords or phrases responsible for a sentence being assigned a particular class or sentiment. Say if I use a BERT model for emotion analysis,is it possible to identify the words or phrases for a sentence having a particular sentiment and changing which would alter their sentiment or semantic essence/meaning.I wonder anyone could shed some light on this as to which layers could be exposed to identify those keywords or if there is any other method to do this.
2021-01-05T08:32:50Z
It sounds to me like you’re trying to do style transfer from one sentiment to another. This is an active area of research currently.One of the most popular papers is linked below but doesn’t use transformers:ACL AnthologyDelete, Retrieve, Generate: a Simple Approach to Sentiment and Style TransferJuncen Li, Robin Jia, He He, Percy Liang. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.And here is a paper that builds on those methods using transformers:arxiv.org2005.12086.pdf338.76 KBBest of luck
2021-01-05T09:37:11Z
FL33TW00D:One of the most popular papers is linked below but doesn’t use transformers:Hi@FL33TW00D. Yes Thank you for sharing this paper the objective is to work towards that goal. I am currently trying to identify the weight of the words in the sentence to that particular sentiment/class. For example I am currently using the bart-large-mnli for zero-shot identification of a class(sentiment/intent) in a sentence.eg: “The service was outstanding but the breakfast was poor”for class labels ‘Food’ or ‘quality’ gives scores of 0.15 and 0.85 respectively. I want to identify the weight of the words in the sentence that contribute to that score of 0.85 forquality(Which primarily would be ‘service’,‘outstanding’,‘breakfast’,‘poor’). I want to know how I can extract this information assuming it is present in the final layers of the model.Thank you for you help
2021-01-05T12:05:42Z
Hi@thomasdaryl,I’ve seen a paper do what you’re looking for, as an extension of the DRG model they use the attention weights of the model in order to identify which tokens are contributing to the classification results most.The paper is seen here:arxiv.org1908.09368.pdf415.77 KBThey also have a helpful git repo here:GitHub - agaralabs/transformer-drg-style-transfer: This repository have scripts and Jupyter-notebooks to perform all the different steps involved in Transforming Delete, Retrieve, Generate Approach for Controlled Text Style TransferWhich specifies that they used BERT to do exactly the task that you’re looking for as I understand it:We have usedBERTfor classification. This classification trainings helps to find the attributes from the sentence. We choose one particular head of BERT model, for which the tokens which have high attention weights are those that are stylistic attributes of the sentence.Hope this helps
2023-04-09T22:40:52Z
Dear Thomas,I have a question regarding your previous work “I am currently trying to identify the weight of the words in the sentence to that particular sentiment/class”.My task is simpler. I have NLP model that does sentiment analysis and I want to know word(s) that has the most influence on the assignment of sentiment (positive, negative, and neutral).Did you publish any manuscript or code on your task?Sincerely,Oleksandr