text
So far I've implemented more accurate 👌 assessment of LLMs reasoning capabilities in Target Sentiment Analysis (zero-shot mode).  With that, recalculated tables of the related benchmark 📊 also has better separation into categories, with the following 🏆 top 🏆 performing models: 
So far I've implemented more accurate 👌 assessment of LLMs reasoning capabilities in Target Sentiment Analysis (zero-shot mode).  With that, recalculated tables of the related benchmark 📊 also has better separation into categories, with the following 🏆 top 🏆 performing models: 
new_line
new_line
text
🟩 1. Proprietary models (🏆 GPT-4  🇺🇸 / GPT-3.5-0613  🇷🇺 )
🟩 1. Proprietary models (🏆 GPT-4  🇺🇸 / GPT-3.5-0613  🇷🇺 )
new_line
text
🟥 2. Open and < 100B (🏆 LLaMA-3-70B)
🟥 2. Open and < 100B (🏆 LLaMA-3-70B)
new_line
text
🟧 3. Open and < 10B (🏆LLaMA-3-8B-Instruct 🇺🇸 / Qwen-2-7B-Instruct 🇷🇺)
🟧 3. Open and < 10B (🏆LLaMA-3-8B-Instruct 🇺🇸 / Qwen-2-7B-Instruct 🇷🇺)
new_line
text
🟨 4. Open and less 1B (🏆Flan-T5-large 🇺🇸 / Qwen2-0.5B-Instruct 🇷🇺)
🟨 4. Open and less 1B (🏆Flan-T5-large 🇺🇸 / Qwen2-0.5B-Instruct 🇷🇺)
new_line
new_line
text
Benchmark: 
Benchmark: 
link
https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
https://github.com/nicolay-r/RuSentNE-LLM-Benchmark
So far I've implemented more accurate 👌 assessment of LLMs reasoning capabilities in Target Sentiment Analysis (zero-shot mode).  With that, recalculated tables of the related benchmark 📊 also has better separation into categories, with the following 🏆 top 🏆 performing models: 

🟩 1. Proprietary models (🏆 GPT-4  🇺🇸 / GPT-3.5-0613  🇷🇺 )
🟥 2. Open and < 100B (🏆 LLaMA-3-70B)
🟧 3. Open and < 10B (🏆LLaMA-3-8B-Instruct 🇺🇸 / Qwen-2-7B-Instruct 🇷🇺)
🟨 4. Open and less 1B (🏆Flan-T5-large 🇺🇸 / Qwen2-0.5B-Instruct 🇷🇺)

Benchmark: https://github.com/nicolay-r/RuSentNE-LLM-Benchmark