text
📢Delighted to share personal findings  🔎 East Asian LLM Qwen1.5 🇨🇳 reasoning capabilities 🧠 Target Sentiment Analysis (TSA).  Starting with the one of the smallest Qwen1.5-1.8B-Chat version, for the original Eastern-European texts (🇷🇺) and their translated versions (🇺🇸) in zero-shot-learning mode setup, the key takeaways of such experiments were as follows:
📢Delighted to share personal findings  🔎 East Asian LLM Qwen1.5 🇨🇳 reasoning capabilities 🧠 Target Sentiment Analysis (TSA).  Starting with the one of the smallest Qwen1.5-1.8B-Chat version, for the original Eastern-European texts (🇷🇺) and their translated versions (🇺🇸) in zero-shot-learning mode setup, the key takeaways of such experiments were as follows:
new_line
new_line
text
✅ 1. Model is capable to perform reasoning in Eastern Eropean languages (🇷🇺) (remember it is 1.8B), switching to Qwen2 results in strong improvement, with results that surpasses LLaMA2-70B-chat (more on difference below). 
✅ 1. Model is capable to perform reasoning in Eastern Eropean languages (🇷🇺) (remember it is 1.8B), switching to Qwen2 results in strong improvement, with results that surpasses LLaMA2-70B-chat (more on difference below). 
new_line
text
✅ 2. Despite the size of 1.8B, reasoning in English has a significant gap in underpeforming (F1=~34%) to the most closest Flan-T5-XL (2.8B) which showcases F1=43%.
✅ 2. Despite the size of 1.8B, reasoning in English has a significant gap in underpeforming (F1=~34%) to the most closest Flan-T5-XL (2.8B) which showcases F1=43%.
new_line
new_line
text
💡 The most intriguing fact that Qwen1.5-1.8B-Chat: 
💡 The most intriguing fact that Qwen1.5-1.8B-Chat: 
new_line
text
it generates new words in Russian I've never seen before: "неретеневая" (negativitively), and imputes the entries in Chinese. The reason of such a low results, is that model was not been able to follow the input instruction and shares all the opinions per each class.  All of that has been improved though in Qwen2-1.5B.
it generates new words in Russian I've never seen before: "неретеневая" (negativitively), and imputes the entries in Chinese. The reason of such a low results, is that model was not been able to follow the input instruction and shares all the opinions per each class.  All of that has been improved though in Qwen2-1.5B.
new_line
new_line
text
Benchmark: 
Benchmark: 
link
https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
new_line
text
Model: 
Model: 
resource
https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat
model
Qwen/Qwen1.5-1.8B-Chat
https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat
new_line
text
Dataset: 
Dataset: 
link
https://github.com/dialogue-evaluation/RuSentNE-evaluation
https://github.com/dialogue-evaluation/RuSentNE-evaluation
new_line
text
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
new_line
text
Collection: 
Collection: 
link
https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
📢Delighted to share personal findings  🔎 East Asian LLM Qwen1.5 🇨🇳 reasoning capabilities 🧠 Target Sentiment Analysis (TSA).  Starting with the one of the smallest Qwen1.5-1.8B-Chat version, for the original Eastern-European texts (🇷🇺) and their translated versions (🇺🇸) in zero-shot-learning mode setup, the key takeaways of such experiments were as follows:

✅ 1. Model is capable to perform reasoning in Eastern Eropean languages (🇷🇺) (remember it is 1.8B), switching to Qwen2 results in strong improvement, with results that surpasses LLaMA2-70B-chat (more on difference below). 
✅ 2. Despite the size of 1.8B, reasoning in English has a significant gap in underpeforming (F1=~34%) to the most closest Flan-T5-XL (2.8B) which showcases F1=43%.

💡 The most intriguing fact that Qwen1.5-1.8B-Chat: 
it generates new words in Russian I've never seen before: "неретеневая" (negativitively), and imputes the entries in Chinese. The reason of such a low results, is that model was not been able to follow the input instruction and shares all the opinions per each class.  All of that has been improved though in Qwen2-1.5B.

Benchmark: https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
Model: https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat
Dataset: https://github.com/dialogue-evaluation/RuSentNE-evaluation
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
Collection: https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
👍
victor kristaller486 louisbrulenaudet ZeroWw