text
📊 Just measured reasoning capabilities 🧠  of Qwen1.5-7B 🇨🇳 in Target Sentiment Analysis (TSA) both for original texts (🇷🇺) and translated in English (🇺🇸), in zero-shot-learning mode.  Here is what I've noticed:
📊 Just measured reasoning capabilities 🧠  of Qwen1.5-7B 🇨🇳 in Target Sentiment Analysis (TSA) both for original texts (🇷🇺) and translated in English (🇺🇸), in zero-shot-learning mode.  Here is what I've noticed:
new_line
text
☑️ 1. Huge gap 📈 with the smaller Qwen1.5 and Qwen2 (1.8B and 1.8B).  Qwen1.5-7B strongly outperforms their "smaller bros" so that case when scale of the model matters.
☑️ 1. Huge gap 📈 with the smaller Qwen1.5 and Qwen2 (1.8B and 1.8B).  Qwen1.5-7B strongly outperforms their "smaller bros" so that case when scale of the model matters.
new_line
text
☑️ 2.  Qwen1.5-7B in english (🇺🇸) behaves similar but slightly underperforming 📉 to the most latest 7B alternatives ... and even including Phi-3-small (3.4B)
☑️ 2.  Qwen1.5-7B in english (🇺🇸) behaves similar but slightly underperforming 📉 to the most latest 7B alternatives ... and even including Phi-3-small (3.4B)
new_line
text
☑️ 3. On texts in (🇷🇺) there is a certain underperforming  📉  gap between the most latest 7B alternatives: F1=34.1, other 7B starts with 40.23.
☑️ 3. On texts in (🇷🇺) there is a certain underperforming  📉  gap between the most latest 7B alternatives: F1=34.1, other 7B starts with 40.23.
new_line
new_line
text
In terms of responses, for non-english texts  (🇷🇺) model answers strict and behaves similar to FlanT5. 
In terms of responses, for non-english texts  (🇷🇺) model answers strict and behaves similar to FlanT5. 
new_line
text
Curious about improvements in Qwen2-7B 🔥
Curious about improvements in Qwen2-7B 🔥
new_line
new_line
text
Model: 
Model: 
resource
https://huggingface.co/Qwen/Qwen1.5-7B-Chat
model
Qwen/Qwen1.5-7B-Chat
https://huggingface.co/Qwen/Qwen1.5-7B-Chat
new_line
text
Benchmark: 
Benchmark: 
link
https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
new_line
text
Dataset: 
Dataset: 
link
https://github.com/dialogue-evaluation/RuSentNE-evaluation
https://github.com/dialogue-evaluation/RuSentNE-evaluation
new_line
text
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
new_line
text
Collection: 
Collection: 
link
https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
📊 Just measured reasoning capabilities 🧠  of Qwen1.5-7B 🇨🇳 in Target Sentiment Analysis (TSA) both for original texts (🇷🇺) and translated in English (🇺🇸), in zero-shot-learning mode.  Here is what I've noticed:
☑️ 1. Huge gap 📈 with the smaller Qwen1.5 and Qwen2 (1.8B and 1.8B).  Qwen1.5-7B strongly outperforms their "smaller bros" so that case when scale of the model matters.
☑️ 2.  Qwen1.5-7B in english (🇺🇸) behaves similar but slightly underperforming 📉 to the most latest 7B alternatives ... and even including Phi-3-small (3.4B)
☑️ 3. On texts in (🇷🇺) there is a certain underperforming  📉  gap between the most latest 7B alternatives: F1=34.1, other 7B starts with 40.23.

In terms of responses, for non-english texts  (🇷🇺) model answers strict and behaves similar to FlanT5. 
Curious about improvements in Qwen2-7B 🔥

Model: https://huggingface.co/Qwen/Qwen1.5-7B-Chat
Benchmark: https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
Dataset: https://github.com/dialogue-evaluation/RuSentNE-evaluation
Related paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
Collection: https://huggingface.co/collections/nicolay-r/sentiment-analysis-665ba391e0eba729021ea101
🔥
victor
/avatars/d7fdc6472f7221dd56af8f3267605133.svg
yaw
megoyaw3
user
https://cdn-avatars.huggingface.co/v1/production/uploads/64e62d11d27a8292c3637f86/aptDeBHpCJxcREj6KPLN1.jpeg
Nicolay Rusnachenko
nicolay-r
user