HairFastGAN: Realistic and Robust Hair Transfer with a Fast
  Encoder-Based Approach (2404.01094)

🚀💇♂🔥 New Research Alert (Avatars Collection)! 🔥💇♀🚀
📄 Title: HairFastGAN: Realistic and Robust Hair Transfer with a Fast Encoder-Based Approach

📝 Description: HairFastGAN is a fast, encoder-based approach to realistic and robust hair transfer that operates in the FS latent space of StyleGAN and includes enhanced in-painting and improved encoders for better alignment, color transfer, and post-processing.

👥 Authors: Maxim Nikolaev, Mikhail Kuznetsov, Dmitry Vetrov, and Aibek Alanov

🔗 Paper: https://huggingface.co/papers/2404.01094

📁 Repository: https://github.com/AIRI-Institute/HairFastGAN

🤗 Demo: https://huggingface.co/spaces/multimodalart/hairfastgan
🔥 Model 🤖: https://huggingface.co/AIRI-Institute/HairFastGAN

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords:  #HairFastGAN #StyleGAN #VirtualTryOn #HairTransfer #AIHairStyling #GenerativeModels #ComputerVision #ImageProcessing #DeepLearning

DmitryRyumin clem FantasiaFoundry multimodalart ameerazam08 tcarteratl victor AlekseiPravdin


# On Coding Your First Attention

While it’s not necessarily the case that you must code the attention block of a transformer from scratch to understand how it works, yet it sure is the closest you can get to having a first-principles understanding of why/how transformers behave the way they do.

@karpathy covered attention in detail in his nanoGPT video (strongly recommend watching). Now I would like to share some thoughts and experience in writing my first attention.

First let’s zoom out quickly and explain what attention is in transformers: Attention in transformers is a communication mechanism that allows the model to focus on different parts of the input sequence when making predictions.

It assigns weights to each input token based on its relevance to the current context, enabling the model to weigh information selectively. This mechanism helps transformers capture long-range dependencies and contextual information effectively.

The official AIAN paper introduced two commonly used forms of attentions: Scaled Dot-Product Attention (also known as Self-Attention) and a stack of self-attention blocks known as Multi-Head Attention.

# The Code

Now, attention as for most deep learning algorithms boils down to a math equation. So writing the code can get really trivial especially with a deep learning framework like PyTorch. Below is what's called a Single Head Attention

(image 2)

The code defines single-head attention in PyTorch - it transforms input vectors, computes attention scores and weights, and then calculates the weighted sum of values based on these weights (as per the attention equation)

When you have multiple of those stacked in parallel, you get what's called Multi-Head Attention. This gives a much simpler code if you are inheriting from the SingleHeadAttention class:

(image 3)

Full Article here: https://huggingface.co/blog/Jaward/coding-your-first-attention

DmitryRyumin Tensorkart Citaman alielfilali01 Theli LeroyDyer martineden


Shining Valiant 2 is here!
Finetuned on Llama 3 70b.

Super excited to build with Llama 3 - more models to come soon :)

https://huggingface.co/ValiantLabs/Llama3-70B-ShiningValiant2


Dynamic Typography: Bringing Words to Life (2404.11614)

Dynamic Typography

Bringing Words to Life

https://huggingface.co/papers/2404.11614

Text animation serves as an expressive medium, transforming static communication into dynamic experiences by infusing words with motion to evoke emotions, emphasize meanings, and construct compelling narratives. Crafting animations that are semantically aware poses significant challenges, demanding expertise in graphic design and animation. We present an automated text animation scheme, termed "Dynamic Typography", which combines two challenging tasks. It deforms letters to convey semantic meaning and infuses them with vibrant movements based on user prompts. Our technique harnesses vector graphics representations and an end-to-end optimization-based framework. This framework employs neural displacement fields to convert letters into base shapes and applies per-frame motion, encouraging coherence with the intended textual concept. Shape preservation techniques and perceptual loss regularization are employed to maintain legibility and structural integrity throughout the animation process. We demonstrate the generalizability of our approach across various text-to-video models and highlight the superiority of our end-to-end methodology over baseline methods, which might comprise separate tasks. Through quantitative and qualitative evaluations, we demonstrate the effectiveness of our framework in generating coherent text animations that faithfully interpret user prompts while maintaining readability.

shellywhen math1761 alielfilali01 Csplk orionriker victor kgourgou

FM-1976 victor kgourgou mokkafe sbhambry


💻 Smoothing the Transition from Service LLM to Local LLM

Imagine your go-to LLM service is down, or you need to use it offline – yikes! This project is all about having that "Plan B" ready to go. Here's LLaMA Duo I've been building with @sayakpaul :

✨ Fine-tune a smaller LLM: We used Hugging Face's alignment-handbook to teach a smaller-sized LLM to mimic my favorite large language model. Think of it as that super-smart AI assistant getting a capable understudy.

🤖 Batch Inference: Let's get that fine-tuned LLM working! My scripts generate lots of text like a champ, and we've made sure things run smoothly even with bigger workloads.

🧐 Evaluation: How well is my small LLM doing? We integrated with the Gemini API to use it as an expert judge – it compares my model's work to the original. Talk about a tough critic!

🪄 Synthetic Data Generation: Need to boost that model's performance? Using Gemini's feedback, we can create even more training data, custom-made to make the LLM better.

🧱 Building Blocks: This isn't just a one-time thing – it's a toolkit for all kinds of LLMOps work. Want to change your evaluation metrics? Bring in models trained differently? Absolutely, let's make it happen.

Why this project is awesome:

💪 Reliability: Keep things running no matter what happens to your main LLM source.
🔒 Privacy: Process sensitive information on your own terms.
🗺 Offline capable: No internet connection? No problem!
🕰 Version Control: Lock in your favorite LLM's behavior, even if the service model changes.

We'm excited to share the code on GitHub. Curious to see what you all think! 👉🏻 https://github.com/deep-diver/llamaduo

sayakpaul sudanenator Csplk Fetchniche ajibawa-2023 bjo163


Generalizable Face Landmarking Guided by Conditional Face Warping (2404.12322)

🚀👩🎤🌟 New Research Alert - CVPR 2024! 🌟👩🎤🚀
📄 Title: Generalizable Face Landmarking Guided by Conditional Face Warping

📝 Description: A new method is proposed to learn a generalizable face landmark that can handle different facial styles, using labeled real faces and unlabeled stylized faces.

👥 Authors: Jiayi Liang, Haotian Liu, Hongteng Xu, Dixin Luo

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2404.12322

🌐 Github Page: https://plustwo0.github.io/project-face-landmarker/
📁 Repository: https://github.com/plustwo0/generalized-face-landmarker

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🔍 Keywords: #FaceLandmarking #DomainAdaptation #FaceWarpping #CVPR2024 #DeepLearning #Innovation


New multimodal dataset by @xai-org @liuhaotian 🤩❤ https://huggingface.co/datasets/xai-org/RealworldQA

qubvel-hf clem talaviyabhavik not-lain Tonic anldrms louisbrulenaudet


The Cauldron is a massive collection of 50 high-quality datasets, all converted to the user/assistant format, and ready to use to fine-tune any Vision Language Model.

The Cauldron covers a wide range of tasks, including general visual question answering, counting, captioning, text transcription, document understanding, chart/figure understanding, table understanding, visual reasoning, geometry, spotting differences between 2 images or converting a screenshot to a code.

https://huggingface.co/datasets/HuggingFaceM4/the_cauldron

VictorSanh Vanessasml victor clem giux78 alexcombessie yo zealota martineden andito

VictorSanh clem fdaudens qubvel-hf yo martineden


Can't wait to see multimodal LLama 3!

We released a resource that might come in handy: The Cauldron 🍯

The Cauldron is a massive manually-curated collection of 50 vision-language sets for instruction fine-tuning. 3.6M images, 30.3M query/answer pairs.

It covers a large variety of downstream uses: visual question answering on natural images, OCR, document/charts/figures/tables understanding, textbooks/academic question, reasoning, captioning, spotting differences between 2 images, and screenshot-to-code.

https://huggingface.co/datasets/HuggingFaceM4/the_cauldron

victor fdaudens InferenceIllusionist qubvel-hf adamelliotfields wannaphong ajibawa-2023 yo martineden AlekseiPravdin alielfilali01


Do I need to make it a tradition to post here every Friday? Well, here we are again!

This week, I'm happy to share that we have two official Mistral models on the Leaderboard! 🔥 You can check them out: https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1 and https://huggingface.co/mistralai/Mixtral-8x22B-v0.1

The most exciting thing here? https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1 model got a first place among pretrained models with an impressive average score of 79.15!🥇 Not far behind is the Mixtral-8x22B-v0.1, achieving second place with an average score of 74.47! Well done, Mistral AI! 👏

Check out my screenshot here or explore it yourself at the https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

The second news is that https://huggingface.co/CohereForAI/c4ai-command-r-plus model in 4-bit quantization got a great average score of 70.08. Cool stuff, Cohere! 😎 (and I also have the screenshot for this, don't miss it)

The last news, which might seem small but is still significant, the Leaderboard frontpage now supports Python 3.12.1. This means we're on our way to speed up the Leaderboard's performance! 🚀

If you have any comments or suggestions, feel free to also tag me on X (Twitter), I'll try to help – [at]ailozovskaya

Have a nice weekend! ✨

clem KonradSzafer Vanessasml clefourrier qubvel-hf SaylorTwift Nacholmo

clem KonradSzafer clefourrier SaylorTwift


Love this new Space built by @enzostvs + @Xenova for Transformers.js: Generate your own AI music (In-browser generation) with AI Jukebox

https://huggingface.co/spaces/enzostvs/ai-jukebox

radames Xenova enzostvs kramp victor qubvel-hf clem grv805 Azamat1k


IMO, the "grounded generation" feature from Cohere's CommandR+ has flown under the radar...

For RAG use cases, responses directly include inline citations, making source attribution an inherent part of generation rather than an afterthought 😎

Who's working on an open dataset with this for the HF community to fine-tune with??

🔗CommandR+ Docs: https://docs.cohere.com/docs/retrieval-augmented-generation-rag

🔗Model on the 🤗 Hub: https://huggingface.co/CohereForAI/c4ai-command-r-plus

ankity09 radames tomaarsen qubvel-hf clem kristaller486 abidlabs

iAkashPaul tomaarsen abidlabs derek-thomas


Unsolvable Problem Detection: Evaluating Trustworthiness of Vision
  Language Models (2403.20331)

A great vision language benchmark: MM-UPD evaluates how model responds to unsolvable problems 🤓

As of now, most VLMs, including GPT-4V and LLaVA-Next-34B, struggle with refusing to answer
Dataset https://huggingface.co/datasets/MM-UPD/MM-UPD
Paper https://huggingface.co/papers/2403.20331

KingNish qubvel-hf AlekseiPravdin raincandy-u anldrms


🦫 We have just released https://huggingface.co/datasets/argilla/Capybara-Preferences in collaboration with Kaist AI (@JW17, @nlee-208) and Hugging Face (@lewtun)

A new synthetic preference dataset built using `distilabel` on top of the awesome https://huggingface.co/datasets/LDJnr/Capybara from @LDJnr

The current dataset combines the already generated alternative completions from https://huggingface.co/datasets/argilla/distilabel-capybara-dpo-7k-binarized, while also adding the remaining ones using the same approach!

Here are some key features on how we built it:

- 🧹 Duplicate removal, keeping the conversation besides the last assistant response, and some slight pre-processing

- 🤖 Generation of alternative completions for the existing conversations (last turn only) with: https://huggingface.co/mlabonne/NeuralBeagle14-7B, https://huggingface.co/argilla/notus-7b-v1, and https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B

- 👨🏻🏫 Running UltraFeedback via GPT-4 to generate the critique i.e. ratings and rationales, for the last assistant responses

- 🎉 Finally, we selected the chosen and rejected responses based on their UltraFeedback score, and applied some slight post-processing!

Sounds simple right? Start building your own synthetic datasets with https://github.com/argilla-io/distilabel already!

lunarflu davanstrien tomaarsen qubvel-hf clem alielfilali01 AtAndDev gabrielmbmb


In the vector search setup, we normally combine a fast embedding model and an accurate but slow reranker model.

The newly released @jinaai rerankers are small in size and almost as accurate as our base reranker. This means given a time constraint, it can scoring more candidate documents from embedding models and have a better chance to feed LLM the correct context for RAG generation.

These models are available on Huggingface and has been integrated into the latest SentenceTransformers 2.7.0. Check it out!

https://huggingface.co/jinaai/jina-reranker-v1-turbo-en
https://huggingface.co/jinaai/jina-reranker-v1-tiny-en

tomaarsen qubvel-hf clem sigridjineth dev7halo ceoofcapybaras


Turns out if you do a cute little hack, you can make https://hf.co/nateraw/musicgen-songstarter-v0.2 work on vocal inputs. 👀

Now, you can hum an idea for a song and get a music sample generated with AI 🔥🔥

Give it a try: ➡ https://hf.co/spaces/nateraw/singing-songstarter ⬅

It'll take your voice and try to autotune it (because let's be real, you're no michael jackson), then pass it along to the model to condition on the melody. It works surprisingly well!

dog bennyzen victor pwab trysem Csplk 2dts dillfrescott manoskary clem lunarflu Nevertree Krognus


🥳 New license for datasets: Apache 2.0!

I have been struggling mentally for many months now with the OpenAI terms of use that indicate that their model outputs cannot be used to build "competing models". This leads to many questions:

- what is the definition of competing? Is it the same as "commercial"?
- since this is part of the terms of use between OpenAI and the API user, can a third party still use the generated dataset to build competing models?
- are such restrictions even legal in the first place?

Trying to "follow the rules" as much as possible despite wanting to be as open as possible, I kept releasing my datasets under non-commercial licenses (which are too restrictive anyhow - nothing should prevent you from using the data in non-LM commercial settings), just like models trained on these datasets. This has put me at a competitive disadvantage compared to creators who do not follow the same approach and release their data/models on apache 2.0 despite the OpenAI "restrictions". Moreover, I fear (https://twitter.com/BramVanroy/status/1780220420316164246) that my approach blocks adaptation of my data/models for (commercial) applications/integrations.

Thankfully @Rijgersberg  noted that these OpenAI terms of use are NOT explicit in the Azure OpenAI API (https://twitter.com/E_Rijgersberg/status/1780308971762450725). Since my latest datasets were created via Azure, this comes as a relief. As far as I can tell after digging through Azure docs, this allows me to change all recent GPT4-generated datasets to apache 2.0! 🥳

- https://huggingface.co/datasets/BramVanroy/ultrachat_200k_dutch
- https://huggingface.co/datasets/BramVanroy/orca_dpo_pairs_dutch
- https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch
- https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch_cleaned
- https://huggingface.co/datasets/BramVanroy/no_robots_dutch

I will have to mull over what I'll do for the older GPT3.5 datasets. What do you think that I should do?


With AutoTrain, you can already finetune the latest llama3 models without writing a single line of code. Here's an example finetune of llama3 8b model: https://huggingface.co/abhishek/autotrain-llama3-no-robots

tinycrops not-lain lunarflu fdaudens xianbao HomerDoh alielfilali01

lunarflu xianbao kramp AtonMountlook alielfilali01 anldrms


Open-source AI on your phone? The HuggingChat app is out for iOs, with the best models: Command R, Zephyr Orpo, Mixtral, Gemma... https://apps.apple.com/ca/app/huggingchat/id6476778843?l=fr-CA

HansWuerst lazarustda BrigitteTousi KingNish samusenps lunarflu pcuenq Nymbo


In a basic chatbots, errors are annoyances. In medical LLMs, errors can have life-threatening consequences 🩸

It's therefore vital to benchmark/follow advances in medical LLMs before even thinking about deployment.

This is why a small research team introduced a medical LLM leaderboard, to get reproducible and comparable results between LLMs, and allow everyone to follow advances in the field.

https://huggingface.co/spaces/openlifescienceai/open_medical_llm_leaderboard

Congrats to  @aaditya and @pminervini !
Learn more in the blog: https://huggingface.co/blog/leaderboard-medicalllm

KonradSzafer qgallouedec BrigitteTousi lunarflu mariagrandury MaziyarPanahi mmhamdy Ramikan-BR

KonradSzafer BrigitteTousi lunarflu fdaudens MaziyarPanahi mmhamdy Ramikan-BR

samusenps lunarflu nicoism mariagrandury MaziyarPanahi Ramikan-BR Pretergeek


🚀 Sentence Transformers v2.7.0 is out! Featuring a new loss function, easier Matryoshka model inference & evaluation, CrossEncoder improvements & Intel Gaudi2 Accelerator support. Details:

1⃣ A new loss function: CachedGISTEmbedLoss
This loss function is a combination of CachedMultipleNegativesRankingLoss and the GISTEmbedLoss, both of which are already excellent. The caching mechanism allows for much higher batch sizes with constant memory usage, which boosts training performance. The GIST part introduces a guide model to guide the in-batch negative sample selection. This prevents false negatives, resulting in a stronger training signal.

2⃣ Automatic Matryoshka model truncation
Matryoshka models produce embeddings that are still useful after truncation. However, this truncation always had to be done manually, until now! We've added a `truncate_dim` option to the Sentence Transformer constructor. This also allows truncation when using `HuggingFaceEmbeddings` from LlamaIndex or LangChain.

3⃣ Additionally, you can now specify `truncate_dim` in evaluators to get the performance after truncation. (Hint: it's surprisingly good, even for models not trained with MatryoshkaLoss, and it can speed up e.g. clustering, retrieval, etc.)

4⃣ CrossEncoder improvements
The CrossEncoder now supports 'push_to_hub' to upload trained reranker models to Hugging Face. Additionally, CrossEncoders now support `trust_remote_code` to load models with custom modelling code.

5⃣ Inference on Intel Gaudi2
If you have an Intel Gaudi2 Accelerator, Sentence Transformers now uses it automatically for even faster inference. No changes are necessary to your code, the device is automatically detected!

Check out the release notes for all of the details: https://github.com/UKPLab/sentence-transformers/releases/tag/v2.7.0

I'm very excited for the upcoming releases: I'm making great progress with a notable v3 refactor that should heavily improve the training process for embedding models!

ajibawa-2023 mdouglas BrigitteTousi lunarflu mrdbourke beomi nickprock alvarobartt icpro MichelleRuwen not-lain Ali-Khaled adityakusupati LeoLee23


🔥 Next Thursday 4/25 at 8am PT / 11am ET / 17h CET,  join our live Hugging Cast to learn how to deploy open models on Google Cloud.

Register ➡ https://streamyard.com/watch/xz2nxp85Pi6e

@philschmid , @tengomucho ,  @jeffboudier will show you brand new Hub integrations built with GCP
🔥 with HF Inference Endpoints
🌎 with Vertex and GKE
🚀 on TPU

ajibawa-2023 BrigitteTousi lunarflu not-lain CYNIC78 n-e-w Ryukijano

Bkarine lunarflu not-lain CYNIC78 Ryukijano


🚀 **InfiniTransformer, Gemma/Llama3 based Implementation!** 🌌

> Update @ 2024.04.19: It now supports Llama-3!

> Note: this implementation is unofficial

This implementation is designed to handle virtually infinite context lengths.

Here's the github repo: https://github.com/Beomi/InfiniTransformer

📄 **Read the original Paper:** https://arxiv.org/abs/2404.07143

## **Focus on Infini-Attention**

- **2 Types of Implementation available:** Attention-layer only implementation / Model & Train-wise implementation
- **Fixed(segment dependent) Memory Usage:** Enables training on larger models and longer sequences without the memory overhead typical of standard Transformer implementations.
- **Infinite Context Capability:** Train with unprecedented sequence lengths—imagine handling up to 1 million sequence lengths on standard hardware!
  - You could train Gemma-2B with 1M sequence length with 2K segmentation size with single H100 GPU.

## **Try InfiniTransformer**

1. **Clone the repository:**
   ```bash
   git clone https://github.com/Beomi/InfiniTransformer
   ```
2. **Install necessary tools:**
   ```bash
   pip install -r requirements.txt
   pip install -e git+https://github.com/huggingface/transformers.git@b109257f4f#egg=transformers
   ```
3. **Dive Deep into Custom Training:**
   - Train with extensive sequence lengths using scripts such as `./train.gemma.infini.noclm.1Mseq.sh`.

for more detailed info, please visit Repo: https://github.com/Beomi/InfiniTransformer

Look forward to see your feedbacks! 😊

ps. Training loss plot is here 😉

tomaarsen maywell lunarflu hllj Joseph717171 genne HyeonjinXZ


Multilingual RAG optimized models and datasets available from the Language Technologies Unit @ the Barcelona Supercomputing Unit
We are releasing new RAG-optimized multilingual models and dataset, within the AINA project contributions:

https://huggingface.co/projecte-aina/FlorRAG , based on out Bloom Flor6.3b model, capable of RAG in Catalan, Spanish and English

https://huggingface.co/datasets/projecte-aina/RAG_Multilingual , a 56K+ instructional dataset with human-like answers created from kernel-of-truth of extractive datasets using a Mixtral8x7b model


Let's breakdown the technical details in Microsoft's mind blowing Lifelike audio-driven talking faces framework - VASA and model VASA-1:

Summary of Summaries
- The paper introduces VASA, a framework for generating lifelike talking faces with appealing visual affective skills (VAS) from a single image and speech audio.
- Core innovations include a diffusion-based model for holistic generation of facial dynamics and head movements in an expressive, disentangled face latent space developed using video data..
- VASA-1 Generates high-quality 512x512 videos at up to 40 FPS with low latency.
- Supports real-time generation of lifelike, emotive talking faces.

Summary of Overall Framework:
- VASA generates facial dynamics and head motion in latent space, conditioned on audio and other signals
- Instead of directly generating video frames, it generates holistic facial dynamics and head motion in a latent space, conditioned on audio and optional signals.
- To achieve this, the framework uses a face encoder-decoder to extract appearance and identity features and train a Diffusion Transformer model to generate motion latent codes.

Technical Method Details:
Expressive and Disentangled Face Latent Space Construction:
   - Based on 3D-AID face reenactment framework
   - Decomposes face into 3D appearance volume, identity code, head pose,
      and facial dynamics latents
   - Uses encoders to extract these latent factors from face images.
   - Applies additional losses to improve disentanglement:
       - Pairwise head pose and facial dynamics transfer loss
       - Face identity similarity loss for cross-identity pose/dynamics transfer

Holistic Facial Dynamics Generation with Diffusion Transformer:
- Represents all facial movements (lip, expression, gaze, etc.) as a single
latent sequence
 - Applies a Diffusion Transformer model to generate the facial dynamics sequence.
- Diffusion Transformer trained with simplified denoising score matching objective.

dashfunnydashdash alvarlaigna SandraToolan pe65374 youngzoo ajibawa-2023 MH0386 lunarflu DmitryRyumin VanshGehlot

clem cncqbaiying Logge victor SandraToolan lunarflu


We posted new SOTA SambaLingo 70B parameter models for Arabic, Thai and Hungarian!

Check out the models here https://huggingface.co/collections/sambanovasystems/sambalingo-65e25770f2037c85ad35ca77

and our paper
https://arxiv.org/abs/2404.05829

Dlbk boapps Violette zolicsaki lunarflu Axilex abidlabs franchb clem ChangranHuuu jlli

lunarflu Axilex abidlabs franchb ChangranHuuu


Diaries of Open Source. Part 15 🤗

🕵♀Idefics 2 is out, a multimodal open-source model with very nice capabilities
Models, demo, and datasets: https://hf.co/collections/HuggingFaceM4/idefics2-661d1971b7c50831dd3ce0fe
Blog: https://hf.co/blog/idefics2

💾Snowflake released snowflake-arctic-embed, a family of powerful small embedding models
Model: https://huggingface.co/Snowflake/snowflake-arctic-embed-m
Blog: https://www.snowflake.com/blog/introducing-snowflake-arctic-embed-snowflakes-state-of-the-art-text-embedding-family-of-models/

✨Pile-T5, EleutherAI's T5 model trained on 2T tokens
Blog: https://blog.eleuther.ai/pile-t5/
Models: https://hf.co/collections/EleutherAI/pile-t5-65a76a0d0022dd270b385a66
GitHub: https://github.com/EleutherAI/improved-t5

🤖CodeQwen1.5-7B base and chat models. Models trained on 3T tokens strong benchmark results for code generation, editing and SQL
Blog post: https://qwenlm.github.io/blog/codeqwen1.5/
Demo: https://hf.co/spaces/Qwen/CodeQwen1.5-7b-Chat-demo
Models: https://hf.co/Qwen/CodeQwen1.5-7B and https://hf.co/Qwen/CodeQwen1.5-7B-Chat

Misc
🦉 DocOwl1.5: Unified Stucture Learning for OCR-free Document Understanding https://hf.co/spaces/mPLUG/DocOwl
👀Cerule - a tiny Vision LM model https://hf.co/Tensoic/Cerule-v0.1
ChemLLM - a LLM for chemistry and molecule science ⚗https://hf.co/AI4Chem/ChemLLM-7B-Chat-1.5-DPO
Distil Whisper Large
📝New pdf/OCR datasets with 19 samples https://hf.co/collections/pixparse/pdf-document-ocr-datasets-660701430b0346f97c4bc628
🔥Gretel AI high quality text-to-sql synthetic dataset https://huggingface.co/datasets/gretelai/synthetic_text_to_sql

DmitryRyumin BrigitteTousi lunarflu not-lain fdaudens Dlbk clem ajibawa-2023 tranhoangnguyen03 dillfrescott clefourrier diwank Farabi224

lunarflu not-lain Dlbk clem pabloce dillfrescott diwank Ollyy

lunarflu not-lain Dlbk clem dillfrescott


PSYDIAL: Personality-based Synthetic Dialogue Generation using Large
  Language Models (2404.00930)

😀🤓😎 New Research Alert - LREC-COLING 2024 (Big Five Personality Traits Collection)! 😎😉😤
📄 Title: PSYDIAL: Personality-based Synthetic Dialogue Generation using Large Language Models

📝 Description: The PSYDIAL presents a novel pipeline for generating personality-based synthetic dialog data to elicit more human-like responses from language models, and presents a Korean dialog dataset focused on personality-based dialog.

👥 Authors: Ji-Eun Han et al.

📅 Conference: LREC-COLING, May 20-25, 2024 | Torino, Italia 🇮🇹

🔗 Paper: https://huggingface.co/papers/2404.00930

🚀 Added to the Big Five Personality Traits Collection: https://huggingface.co/collections/DmitryRyumin/big-five-personality-traits-661fb545292ab3d12a5a4890

🔥🔝 See also OCEAN-AI - https://huggingface.co/spaces/ElenaRyumina/OCEANAI (App, co-authored by @DmitryRyumin) 😉

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🔍 Keywords: #PSYDIAL #PersonalityDialogues #SyntheticData #LanguageModels #ConversationalAI #KoreanDialogues #BigFivePersonality #ExtraveresionDialogues #OCEANAI #BigFive #PersonalityTraits #PersonalityAnalysis #LREC-COLING2024 #DeepLearning #Innovation

DmitryRyumin osanseviero BrigitteTousi lunarflu Dlbk DESUCLUB KvrParaskevi thinhlpg

DmitryRyumin BrigitteTousi lunarflu Dlbk clem


I see you all send your documents to close-source APIs, this is not ok 👎 it breaks my heart 💔
I have seen many open-source document models, and I am amazed by what IDEFICS2 has done with document understanding 🤯🤩 it's not something you've ever seen before! https://huggingface.co/spaces/HuggingFaceM4/idefics-8b

Please use it! Has Apache 2.0 license ❤

DmitryRyumin YaTharThShaRma999 victor Masa-Erland osanseviero nkasmanoff BrigitteTousi lunarflu fdaudens Ryukijano Dlbk samusenps clem matlok VictorSanh CookieMaster Tonic not-lain

lunarflu Ryukijano clem raghavprabhakar CookieMaster jbilcke Tonic not-lain


📣 I'm thrilled to announce "ALERT: A Comprehensive #Benchmark for Assessing #LLMs’ Safety through #RedTeaming" 🚨

📄 Paper: https://arxiv.org/pdf/2404.08676.pdf
🗃 Repo: https://github.com/Babelscape/ALERT
🤗 ALERT benchmark: https://huggingface.co/datasets/Babelscape/ALERT
🤗 ALERT DPO data: https://huggingface.co/datasets/Babelscape/ALERT_DPO

As a key design principle for ALERT, we developed a fine-grained safety risk taxonomy (Fig. 2). This taxonomy serves as the foundation for the benchmark to provide detailed insights about a model’s weaknesses and vulnerabilities as well as inform targeted safety enhancements 🛡

For collecting our prompts, we started from the popular
Anthropic's HH-RLHF data, and used automated strategies to filter/classify prompts. We then designed templates to create new prompts (providing sufficient support for each category, cf. Fig. 3) and implemented adversarial attacks.

In our experiments, we extensively evaluated several open- and closed-source LLMs (e.g. #ChatGPT, #Llama and #Mistral), highlighting their strengths and weaknesses (Table 1).

For more details, check out our preprint: https://arxiv.org/pdf/2404.08676.pdf 🤓

Huge thanks to @felfri, @PSaiml, Kristian Kersting, @navigli, @huu-ontocord and @BoLi-aisecure (and all the organizations involved: Babelscape, Sapienza NLP, TU Darmstadt, Hessian.AI, DFKI, Ontocord.AI, UChicago and UIUC)🫂

victor clefourrier osanseviero BrigitteTousi lunarflu huu-ontocord Dlbk tomaarsen AdinaY KvrParaskevi eliolio vicgalle kklyman taufiqdp Vlansu riccorl

felfri lunarflu huu-ontocord Dlbk clem tomaarsen


Contamination free code evaluations with LiveCodeBench! 🖥

LiveCodeBench is a new leaderboard, which contains:
- complete code evaluations (on code generation, self repair, code execution, tests)
- my favorite feature: problem selection by publication date 📅

This feature means that you can get model scores averaged only on new problems out of the training data. This means... contamination free code evals! 🚀

Check it out!

Blog: https://huggingface.co/blog/leaderboard-livecodebench
Leaderboard: https://huggingface.co/spaces/livecodebench/leaderboard

Congrats to @StringChaos  @minimario  @xu3kev @kingh0730 and @FanjiaYan for the super cool leaderboard!

loubnabnl osanseviero BrigitteTousi lunarflu Dlbk clem mathiasn1 kingh0730 thomwolf dillfrescott

lunarflu Dlbk clem kingh0730 thomwolf dillfrescott


Is is time for the open-source AI robots revolution 🚀?

With @haixuantao and @Leyo we’ve been playing with a low-cost DJI robot controlled by three local open-source AI models (Whisper, Idefics2, Parler-TTS - all Apache2) and orchestrated by Dora-cs.

Links to find all the hardware/software we used in the demo:
- robot control framework – dora-rs: https://github.com/dora-rs/dora
- speech-to-text model – whisper: https://huggingface.co/openai/whisper-base
- vision-text model – Idefics2: https://huggingface.co/HuggingFaceM4/idefics2-8b-AWQ
-  text-to-speech model – ParlerTTS mini: https://huggingface.co/parler-tts/parler_tts_mini_v0.1
- robot: https://dji.com/robomaster-s1
- code gist: https://gist.github.com/haixuanTao/860e1740245dc2c8dd85b496150a9320
- Larger codebase: https://huggingface.co/datasets/dora-rs/dora-idefics2
- laptop/pc: any with a recent GPU card (our has a RTX 4090)

Enjoy!

SivilTaram KingNish Tonic clefourrier YaTharThShaRma999 victor osanseviero radames BrigitteTousi lunarflu Leyo giux78 clem Violette dodin InferenceIllusionist Abouharga Martins6 jme2791

Tonic clefourrier YaTharThShaRma999 osanseviero radames BrigitteTousi lunarflu Leyo giux78 clem Violette InferenceIllusionist Martins6 carloscardenas KingNish louisbrulenaudet

Tonic YaTharThShaRma999 osanseviero BrigitteTousi lunarflu giux78 clem Violette TommyZQ OmbelineM


🆕 Evaluate your RL agents - who's best at Atari?🏆

The new RL leaderboard evaluates agents in 87 possible environments (from Atari 🎮 to motion control simulations🚶and more)!

When you submit your model, it's run and evaluated in real time - and the leaderboard displays small videos of the best model's run, which is super fun to watch! ✨

Kudos to @qgallouedec for creating and maintaining the leaderboard!
Let's find out which agent is the best at games! 🚀

https://huggingface.co/spaces/open-rl-leaderboard/leaderboard

qgallouedec osanseviero BrigitteTousi lunarflu clem KvrParaskevi mlabonne


Excited to share that our paper: "AutoAgents: A Framework for Automatic Agent Generation" got accepted to this year's @ IJCAI  🎉🥳

As a young aspiring AI researcher, this one means a lot, as it is the first ever paper I was blessed to contribute in. Thanks to the incredibly brilliant minds I got to work with (Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Börje F. Karlsson, Jie Fu, Yemin Shi ) - you’re heroes of mine 🫡

@karpathy I hope this is worth a mutual follow/response haha, your lessons helped shaped my understanding of this field and they still are. Thank  you 🙏🏼

Demo: https://huggingface.co/spaces/LinkSoul/AutoAgents
Code: https://github.com/Link-AGI/AutoAgents
Paper: https://arxiv.org/abs/2309.17288


We noticed that all the open-source models and datasets from https://huggingface.co/WizardLM in their personal Hugging Face account & in the Microsoft Hugging Face organization (https://huggingface.co/microsoft) have been made private by the author, which will lead some demos to fail (these models were collectively downloaded over a hundred thousand times a month).

This is the explanation that @WizardLM communicated a few hours ago: https://huggingface.co/posts/WizardLM/329547800484476#661e0d17bca1a6038b60503e

We apologize for the inconvenience & are trying to get in touch with the author & Microsoft in order to try to find a good resolution for community members. Let us know if you have any questions!

osanseviero fdaudens ijohn07 pierrci BrigitteTousi lazarustda samusenps clefourrier lunarflu Diz187 guilherme-am m-ric acamilogg88

osanseviero pierrci Mehrdadgpt lunarflu fynnkroeger m-ric


Learn Your Reference Model for Real Good Alignment (2404.09656)

Learn Your Reference Model for Real Good Alignment

https://huggingface.co/papers/2404.09656

The complexity of the alignment problem stems from the fact that existing methods are unstable. Researchers continuously invent various tricks to address this shortcoming. For instance, in the fundamental Reinforcement Learning From Human Feedback (RLHF) technique of Language Model alignment, in addition to reward maximization, the Kullback-Leibler divergence between the trainable policy and the SFT policy is minimized. This addition prevents the model from being overfitted to the Reward Model (RM) and generating texts that are out-of-domain for the RM. The Direct Preference Optimization (DPO) method reformulates the optimization task of RLHF and eliminates the Reward Model while tacitly maintaining the requirement for the policy to be close to the SFT policy. In our paper, we argue that this implicit limitation in the DPO method leads to sub-optimal results. We propose a new method called Trust Region DPO (TR-DPO), which updates the reference policy during training. With such a straightforward update, we demonstrate the effectiveness of TR-DPO against DPO on the Anthropic HH and TLDR datasets. We show that TR-DPO outperforms DPO by up to 19%, measured by automatic evaluation with GPT-4. The new alignment approach that we propose allows us to improve the quality of models across several parameters at once, such as coherence, correctness, level of detail, helpfulness, and harmlessness.


LM Transparency Tool: Interactive Tool for Analyzing Transformer
  Language Models (2404.07004)

Information Flow Routes: Automatically Interpreting Language Models at
  Scale (2403.00824)

🔍 Today's pick in Interpretability & Analysis of LMs: https://huggingface.co/papers/2404.07004 by @igortufanov @mahnerak @javifer @lena-voita

The LLM transparency toolkit is an open source toolkit and visual interface to efficiently identify component circuits in LMs responsible for their predictions, using the Information Flow Routes approach (https://huggingface.co/papers/2403.00824).

The tool enables fine-grained customization, highlighting the importance of individual FFN neurons and attention heads. Moreover, vocabulary projections computed using the logit lens approach are provided to examine intermediate predictions of the residual stream, and tokens promoted by specific component updates.

💻 Code: https://github.com/facebookresearch/llm-transparency-tool

🚀 Demo: https://huggingface.co/spaces/facebook/llm-transparency-tool-demo

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9


FiftyOne Datasets <> Hugging Face Hub Integration!

As of yesterday's release of FiftyOne `0.23.8`, the FiftyOne open source library for dataset curation and visualization is now integrated with the Hugging Face Hub!

You can now load Parquet datasets from the hub and have them converted directly into FiftyOne datasets. To load MNIST, for example:

```bash
pip install -U fiftyone
```

```py
import fiftyone as fo
import fiftyone.utils.huggingface as fouh

dataset = fouh.load_from_hub(
    "mnist",
    format="ParquetFilesDataset",
    classification_fields="label",
)
session = fo.launch_app(dataset)
```

You can also load FiftyOne datasets directly from the hub. Here's how you load the first 1000 samples from the VisDrone dataset:

```py
import fiftyone as fo
import fiftyone.utils.huggingface as fouh

dataset = fouh.load_from_hub("jamarks/VisDrone2019-DET", max_samples=1000)

# Launch the App
session = fo.launch_app(dataset)
```

And tying it all together, you can push your FiftyOne datasets directly to the hub:

```py
import fiftyone.zoo as foz
import fiftyone.utils.huggingface as fouh

dataset = foz.load_zoo_dataset("quickstart")
fouh.push_to_hub(dataset, "my-dataset")
```

Major thanks to @tomaarsen @davanstrien @severo @osanseviero and @julien-c for helping to make this happen!!!

Full documentation and details here: https://docs.voxel51.com/integrations/huggingface.html#huggingface-hub

severo brianmoore davanstrien ehofesmann jguerrero harpreetsahota victor clem merve osanseviero lunarflu julien-c dgural not-lain jamarks shamy777 Maralk

brianmoore davanstrien ehofesmann jguerrero harpreetsahota jamarks victor clem osanseviero lunarflu julien-c dgural not-lain

brianmoore davanstrien ehofesmann jguerrero harpreetsahota victor lunarflu julien-c dgural not-lain jamarks

ehofesmann jguerrero harpreetsahota lunarflu julien-c dgural not-lain jamarks

samusenps lunarflu dgural not-lain jamarks


Demo for IDEFICS-8B demo is out! https://huggingface.co/spaces/HuggingFaceM4/idefics-8b

This checkpoint is not optimized to chat, but rather works very well for various tasks, incl visual question answering and document tasks 💬📑
Chatty one is coming soon!

radames kramp clem Redox3275 osanseviero CristianJD lunarflu VictorSanh


Introducing Super RAGs in Mistral 8x7B-v1 (2404.08940)

Super RAGs in Mistral 8x7B-v1 -
The recent paper on arXiv introduces Super Retrieval-Augmented Generation (Super RAGs), a groundbreaking approach to improve Large Language Models (LLMs) by integrating external knowledge sources. This integration into the Mistral 8x7B v1 LLM has shown notable improvements in accuracy, speed, and user satisfaction.

What are your thoughts on the potential of Super RAGs to transform the future of AI and LLMs?

https://huggingface.co/papers/2404.08940

victor clem merve lunarflu not-lain Jason-Lu

raghav2543 clem lunarflu not-lain Jason-Lu


Just had the former chief public health officer of the Netherlands🇳🇱 review a huggingface AI-doctor I made via a simple orpo-zephyr-8x22b-GPT  and they think it's really good.

https://hf.co/chat/assistant/661d77310e3aea9ae571e43c

pierrci pankaj-munde mbilalkhaan kramp jpodivin m-ric clem mdadnan lunarflu Violette BrigitteTousi jbilcke-hf dillfrescott kellykellz akyarborough andrewgleave surajp bbooth dillen-hub


We release Idefics2-8B, a foundation vision language model with SOTA results for its size on many benchmarks.

For Idefics2, we adopted a simple architecture:
-Images are fed to a vision encoder, then to a modality projection to match the input dimension of the LLM, and finally to a perceiver resampler for efficient pooling.
-Interleaved image-text data are then passed to the LLM.

During the pre-training:
-The modality projection and perceiver resampler weights are newly initialized.
-We start with pre-trained models for the vision encoder and the LLM, and continue the training with LoRA.
-In total, we see 1.5T images!

We pre-train on 3 types of data, all publicly available:
-Interleaved image-text documents: our dataset OBELICS https://huggingface.co/datasets/HuggingFaceM4/OBELICS
-Image caption pairs: only synthetic captions!
-PDF documents: IDL and PDFA

We kept the aspect ratio of the images with the Patch n' Pack strategy, with a resolution of up to 980x980.
At inference, it's also more efficient for lower-resolution images.

For the SFT, we build The Cauldron, a collection of 50 high-quality datasets in the user/assistant format.
It is a ready-to-use dataset for the fine-tuning of any VLM.
https://huggingface.co/datasets/HuggingFaceM4/the_cauldron

Most current models, like LLaVA-NeXT, encode images with an excessive number of tokens, like 2880.
Instead, we put a focus on being efficient at inference by training on a mix of images encoded with 64 tokens, and 320 tokens.
The result is that we perform favorably compared to the best models in our size class, while being efficient at inference.

VictorSanh victor danaaubakirova sbarman25 m-ric clem lunarflu Violette fdaudens sbrandeis

m-ric clem samusenps Sylvestre Molbap NePe lunarflu fynnkroeger Violette

WenchuanYang clem samusenps lunarflu Violette


New open multimodal model in town: Idefics2!

💪 Strong 8B-parameters model: often on par with open 30B counterparts.
🔓Open license: Apache 2.0.
🚀 Strong improvement over Idefics1: +12 points on VQAv2, +30 points on TextVQA while having 10x fewer parameters.
📚 Better data: boosting OCR capabilities with 6TB of documents to transcribe, and improving QA capabilities on charts/figures/diagrams.
🕵♀ Transparent training data: inspect and build upon all the data (10s of TB of data) we trained on.
🔲 More natural image processing: Incorporating strategies to treat images in their native resolution and native aspect ratio.
📸 High-resolution images: image resolutions up to 980 x 980 and integrating strategies that allow to trade computational efficiency for performance.
😎 2 checkpoints: Releasing both base checkpoint and instruction fine-tuned checkpoint. Chat version to come.

Ressources: https://huggingface.co/collections/HuggingFaceM4/idefics2-661d1971b7c50831dd3ce0fe
Blogpost: https://huggingface.co/blog/idefics2

VictorSanh rwightman monsoon-nlp oceansweep pierrci ajibawa-2023 osanseviero Ayushnangia andrewrreed m-ric clem Molbap lunarflu fynnkroeger marcsun13 raincandy-u jlzhou

VictorSanh clem oceansweep pierrci samusenps osanseviero andrewrreed m-ric Molbap lunarflu

VictorSanh clem samusenps andrewrreed m-ric lunarflu


🔥🔥🔥 Introducing WizardLM-2!

📙Release Blog: https://wizardlm.github.io/WizardLM2
✅Model Weights: https://huggingface.co/collections/microsoft/wizardlm-661d403f71e6c8257dbd598a
🐦Twitter: https://twitter.com/WizardLM_AI/status/1779899325868589372

We introduce and opensource WizardLM-2, our next generation state-of-the-art large language models, which have improved performance on complex chat, multilingual, reasoning and agent. New family includes three cutting-edge models: WizardLM-2 8x22B, WizardLM-2 70B, and WizardLM-2 7B.

WizardLM-2 8x22B is our most advanced model, and the best opensource LLM in our internal evaluation on highly complex tasks. WizardLM-2 70B reaches top-tier reasoning capabilities and is the first choice in the same size. WizardLM-2 7B is the fastest and achieves comparable performance with existing 10x larger opensource leading models.

🤗 WizardLM 2 Capacities:

1. MT-Bench (Figure-1)
The WizardLM-2 8x22B even demonstrates highly competitive performance compared to the most advanced proprietary works such as GPT-4-Trubo and Glaude-3. Meanwhile, WizardLM-2 7B and WizardLM-2 70B are all the top-performing models among the other leading baselines at 7B to 70B model scales.

2. Human Preferences Evaluation (Figure 2)
Through this human preferences evaluation, WizardLM-2's capabilities are very close to the cutting-edge proprietary models such as GPT-4-1106-preview, and significantly ahead of all the other open source models.

🔍Method Overview:
As the natural world's human-generated data becomes increasingly exhausted through LLM training, we believe that: the data carefully created by AI and the model step-by-step supervised by AI will be the sole path towards more powerful AI.

In the past one year, we built a fully AI powered synthetic training system. (As shown in the Figure 3).

WizardLM tangmen victor vaclavkosar s0me-0ne oceansweep Bruno xianbao haipeng1 syslot Lin-Rexter Chaklam JosMax veeragoni CocoSun ChuckMcSneed osanseviero julien-c Ramikan-BR niclaslindqvist radames taher30 jinkazamapro07 m-ric jayjay5230 Jaward latent-variable jameshwade Revs64 JSCPPProgrammer cppowboy ShaneTian jlzhou alesthehuman bardusco-tau mahirzukic2 noobmaster29 Emigmo KvrParaskevi Aressions

haipeng1 Lin-Rexter osanseviero julien-c radames kramp latent-variable djester alesthehuman bardusco-tau mahirzukic2 noobmaster29 c4wrd wucl usholanb bs82 mitra-mir

sparkycollier haipeng1 Lin-Rexter YueJie osanseviero julien-c ragibhasan radames latent-variable sugatoray alesthehuman mahirzukic2 noobmaster29

KyleLyu djester alesthehuman pharmaia noobmaster29 tobitege StudentWen VDBLOI2024

Tenbo fysp latent-variable alesthehuman noobmaster29


😀🤓😎 New Space - OCEAN-AI (App, co-authored by @DmitryRyumin) 😎😉😤
🚀 Title: OCEAN-AI is an open-source app for Big Five personality traits assessment and HR-processes automatization.

🤗 Demo: https://huggingface.co/spaces/ElenaRyumina/OCEANAI

👥 Authors: @ElenaRyumina, @DmitryRyumin, and Alexey Karpov

📝 Description: OCEAN-AI consists of a set of modules for intellectual analysis of human behavior based on multimodal data for automatic personality traits (PT) assessment. The app evaluates five PT: Openness to experience, Conscientiousness, Extraversion, Agreeableness, Non-Neuroticism.

The App solves practical tasks:
- Ranking of potential candidates by professional responsibilities.
- Forming effective work teams.
- Predicting consumer preferences for industrial goods.

🔍 Keywords: #OCEANAI #BigFive #PersonalityTraits #PersonalityAnalysis #MultimodalData #Transformers #FirstImpressionsV2 #DeepLearning #Innovation #BehaviorAnalysis #AffectiveRecognition #TeamFormation #ConsumerPreferences #CandidateRanking

DmitryRyumin ElenaRyumina Mihaj nilacadenas arturoxdev MiSTe-R Ramikan-BR kramp

DmitryRyumin ElenaRyumina Mihaj MiSTe-R beno-rohman


𝐀𝐝𝐝𝐢𝐭𝐢𝐨𝐧𝐚𝐥 𝐢𝐧𝐬𝐢𝐠𝐡𝐭𝐬 💡

𝐀𝐝𝐝𝐢𝐭𝐢𝐨𝐧𝐚𝐥 𝐢𝐧𝐬𝐢𝐠𝐡𝐭𝐬 💡

𝐏𝐚𝐩𝐞𝐫 𝐑𝐞𝐯𝐢𝐞𝐰: 𝐑𝐡𝐨-𝟏 - 𝐃𝐨 𝐧𝐨𝐭 𝐮𝐬𝐞 𝐚𝐥𝐥 𝐭𝐨𝐤𝐞𝐧𝐬 𝐞𝐪𝐮𝐚𝐥𝐥𝐲 𝐢𝐧 𝐲𝐨𝐮𝐫 𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠! ⚖⛔

A new paper topping Daily papers questions a hidden assumption in LLM training:

🤔 𝙎𝙝𝙤𝙪𝙡𝙙 𝙬𝙚 𝙧𝙚𝙖𝙡𝙡𝙮 𝙪𝙨𝙚 𝙖𝙡𝙡 𝙩𝙤𝙠𝙚𝙣𝙨 𝙚𝙦𝙪𝙖𝙡𝙡𝙮 𝙞𝙣 𝙤𝙪𝙧 𝙇𝙇𝙈'𝙨 𝙩𝙧𝙖𝙞𝙣𝙞𝙣𝙜 ?

Some tokens are more relevant than others, and some are mostly noise (just look up the history of 𝘚𝘰𝘭𝘪𝘥𝘎𝘰𝘭𝘥𝘔𝘢𝘨𝘪𝘬𝘢𝘳𝘱).

So this paper introduces 𝗦𝗲𝗹𝗲𝗰𝘁𝗶𝘃𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝗶𝗻𝗴, which is actually really simple:
➡ A specific metric measures the relevance of each token. Then during training, only the top k% tokens for this relevance metric count in the loss calculation.

Authors test this method by training models on the difficult MATH dataset (only competition mathematics problems).

➡ Their technique seems like a new must-do in LLM training: Training is much faster and reaches an impressive performance!

𝐑𝐞𝐬𝐮𝐥𝐭𝐬:
◆ ⏱ Training is x5 to x10 faster to reach equivalent performance compared to standard language modeling.
◆ 💪 Their 1B model achieves close to GPT4 Chain-of-Thought performance on MATH!
◆ 🚀 Their 7B model match performance of the state-of-the-art DeepSeek for the same size, while trained on only 3% of tokens

𝐀𝐝𝐝𝐢𝐭𝐢𝐨𝐧𝐚𝐥 𝐢𝐧𝐬𝐢𝐠𝐡𝐭𝐬 💡
◆ Datasets used for pre-training, even after pre-filtering, still contain a large proportion of noisy tokens 😖
◆ Authors show that when you reduce loss on noisy tokens, you actually reduce accuracy (Figure 7). So Selective Language Modeling seems fundamental! ✅

Find great reads in @akhaliq 's Daily Papers 👉 https://huggingface.co/papers
Paper added to my collection 👉 https://huggingface.co/collections/m-ric/spinning-up-in-llms-659e698f9dd5a71bd3f579a7


We just released a new MoE model (meraGPT/mera-mix-4x7B) that is half as large as Mixtral-8x7B while still been competitive with it across different benchmarks. mera-mix-4x7B achieves 76.37 on the open LLM eval.

You can check mera-mix-4x7B out on HF here - https://huggingface.co/meraGPT/mera-mix-4x7B


ORPO: Monolithic Preference Optimization without Reference Model (2403.07691)

Could more DPO-style preference data be crucial for enhancing open LLMs across different languages?

Leveraging a 7k preference dataset Argilla (@alvarobartt), Hugging Face (@lewtun) and Kaist AI (@JW17 & @nlee-208)
utilized Kaist AI's recently introduced ORPO technique https://huggingface.co/papers/2403.07691 with the latest MistralAI MOE model to create a very high-performing open LLM: https://huggingface.co/HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1

Since ORPO doesn't require a separate SFT stage, all that is needed is a strong base model + high-quality DPO-style datasets.

Currently, there is a significant lack of non-English DPO datasets. Filling this gap could significantly improve open LLMs in various languages.

You can get an overview of the current state of DPO datasets across different languages here: https://huggingface.co/spaces/DIBT/preference_data_by_language

JW17 alvarobartt osanseviero nlee-208 NMontanaBrown Violette Heng666 raincandy-u DataSoul


On Agentic AI: Autonomy Is All You Need!

There is this remarkable beauty in witnessing an AI system autonomously completes complex tasks with a level of brilliance that supersedes our reasoning capabilities/expectations - It is the holy grail of creation.

Giving your AI agents autonomy is analogous to us having "free will" and everything else thereafter is a cascade of possibilities and potentials waiting to unfold.

As brilliantly said by @AndrewNg  "It’s a beautiful thing when you see an agent autonomously decide to do things in ways that you had not anticipated, and succeed as a result!"

Autonomy in Agentic AI
 - augments agents' decision-making capabilities.
 - enables adaptation to diverse environments.
 - facilitates real-time learning and improvement.
 - fosters dynamic multi-agent collaboration.
 - promotes efficient and independent task execution.
 - drives innovation in dynamic and unpredictable
    scenarios.

This what the AutoAgents paper conveyed - A fully autonomous agentic framework that basically gives agents the free will for authentic/compelling creativity.

With just three dynamically predefined agents [ Planner, Agent Observer and Plan Observer ] acting collaboratively they are able to generically create autonomous agents:

```
Agent (A) =  {
     Prompt (P) - defines agent's identity fully ,
     Description (D) - adds specific role identity,
     Toolset (T) - equips the agent with tools,
     Suggestion (S) - offers task execution tips
 }
```

Demo: https://huggingface.co/spaces/LinkSoul/AutoAgents
Code: https://github.com/Link-AGI/AutoAgents
Paper: https://arxiv.org/abs/2309.17288


Fun dataset added last week by  @esind from https://huggingface.co/Anthropic to compare persuasiveness between AI and human outputs:
https://huggingface.co/datasets/Anthropic/persuasion

Jaward kgourgou osanseviero not-lain KvrParaskevi InferenceIllusionist nogully


We're introducing experimental support for `device_map` in Diffusers 🤗

If you have multiple GPUs you want to use to distribute the pipeline models, you can do so. Additionally, this becomes more useful when you have multiple low-VRAM GPUs.

Documentation:
https://huggingface.co/docs/diffusers/main/en/training/distributed_inference#device-placement

🚨 Currently, only "balanced" device mapping strategy is supported.

ajibawa-2023 clem osanseviero Bruno Paolo16 alvarobartt ingTikna


Watch the full tutorial here : https://youtu.be/0t5l6CP9eBg

The tutorial is over 2 hours literally with manually fixed captions and perfect video chapters.

Most Awaited Full Fine Tuning (with DreamBooth effect) Tutorial Generated Images - Full Workflow Shared In The Comments - NO Paywall This Time - Explained OneTrainer - Cumulative Experience of 16 Months Stable Diffusion

In this tutorial, I am going to show you how to install OneTrainer from scratch on your computer and do a Stable Diffusion SDXL (Full Fine-Tuning 10.3 GB VRAM) and SD 1.5 (Full Fine-Tuning 7GB VRAM) based models training on your computer and also do the same training on a very cheap cloud machine from MassedCompute if you don't have such computer.

Tutorial Readme File ⤵
https://github.com/FurkanGozukara/Stable-Diffusion/blob/main/Tutorials/OneTrainer-Master-SD-1_5-SDXL-Windows-Cloud-Tutorial.md

Register Massed Compute From Below Link (could be necessary to use our Special Coupon for A6000 GPU for 31 cents per hour) ⤵
https://bit.ly/Furkan-Gözükara

Coupon Code for A6000 GPU is : SECourses

0:00 Introduction to Zero-to-Hero Stable Diffusion (SD) Fine-Tuning with OneTrainer (OT) tutorial
3:54 Intro to instructions GitHub readme
4:32 How to register Massed Compute (MC) and start virtual machine (VM)
5:48 Which template to choose on MC
6:36 How to apply MC coupon
8:41 How to install OT on your computer to train
9:15 How to verify your Python, Git, FFmpeg and Git installation
12:00 How to install ThinLinc and start using your MC VM
12:26 How to setup folder synchronization and file sharing between your computer and MC VM
13:56 End existing session in ThinClient
14:06 How to turn off MC VM
14:24 How to connect and start using VM
14:41 When use end existing session
16:38 How to download very best OT preset training configuration for SD 1.5 & SDXL models
18:00 How to load configuration preset
18:38 Full explanation of OT configuration and best hyper parameters for SDXL
.
.
.

rafalwiatrowski mehdicode123 samusenps kadirnar HeavyLvy clem Aakashk Matix200 AlexSonar levulinh


Here's a utility component for integrating your Gradio app with Hugging Face. This custom component enables you to search for models, spaces, datasets, and users.

`pip install gradio_huggingfacehub_search`

You can see it in action here. https://huggingface.co/spaces/arcee-ai/mergekit-config-generator

And learn how to use it here https://huggingface.co/spaces/radames/gradio_huggingfacehub_search

samusenps osanseviero srisree clem Gevennou multimodalart


"How expensive is it actually to teach a #LanguageModel German through #finetuning 💰💰💰? We get asked this quite often.

There is no one-size-fits-all answer to this question, as among other factors:
⏹ each fine-tuning is different,
⏹ the hardware used can be a major cost driver,
⏹ the amount and type of training data can extend the process,
⏹ and the skills to be trained can increase the difficulty of fine-tuning.

However, we have broken down the costs incurred for our latest fine-tune ( https://huggingface.co/VAGOsolutions/SauerkrautLM-Qwen-32b)

Base model: https://huggingface.co/Qwen/Qwen1.5-32B
Fine-Tuning Goal: Train German language
Training dataset size: 160,000 SFT data / 110,000 DPO data
Training duration: 72.5 hours (2 epochs SFT / 1 epoch DPO)
GPU: 2x A100 SXM
New model: https://huggingface.co/VAGOsolutions/SauerkrautLM-Qwen-32b

Total cost: 312 euros 💶

These are quite reasonable training costs considering the model now speaks passable German (previously very broken). Depending on the use case and process requirements, this can even be a real alternative to the costly continuous pre-training of foreign language models.

jed-tiotuico osanseviero clem victor fpreiss philipp-zettl joennlae Kearm


Yesterday, xAI announced Grok-1.5 Vision - https://x.ai/blog/grok-1.5v. But more importantly, they also released a new VLM benchmark dataset - RealWorldQA. The only problem was that they released it as a ZIP archive. I fixed that! Now you can use it in your evaluations as a regular HF dataset: https://huggingface.co/datasets/visheratin/realworldqa

musfiqdehan fynnkroeger osanseviero Bruno ajibawa-2023 VictorSanh clem not-lain ShihHong alielfilali01 dinhanhx


Introducing MusicGen Web: AI-powered music generation directly in your browser, built with 🤗 Transformers.js! 🎵

Everything runs 100% locally, meaning there are no calls to an API! 🤯 Since it's served as a static HF space, it costs $0 to host and run! 🔥

We also added the ability to share your generated music to the discussion tab, so give it a try! 👇
https://huggingface.co/spaces/Xenova/musicgen-web

DmitryRyumin adamelliotfields samusenps andrewmilesmurphy1992 Lyte rocca Sword666 ajibawa-2023 ashwinlimaye Retr0REG cdosoftei Cksampa osanseviero clem AlekseiPravdin victor ylacombe alt2023 DataSoul b4s36t4 Noomam nekomeowww

luoweiqiang9527 cdosoftei raincandy-u Cksampa osanseviero clem victor Kce niwram69


Spent the morning designing a super important product: SMALL LANGUAGE MODEL BABY ONESIE https://www.shop.hackernoon.com/all-merch/p/small-language-model-baby-onesie

samusenps visheratin travisking Violette


Hi everyone! I lead a team of students to create a game board community of LLMs that interact to help one of the LLMs create a presentation on an idea.
https://github.com/GooseCube/WWPD.ai
This was based on Standfords unusable initial version:

"A group of researchers at Stanford University and Google have created a miniature RPG-style virtual world similar to The Sims, where 25 characters, controlled by ChatGPT and custom code, live out their lives independently with a high degree of realistic behavior. They wrote about their experiment in a preprint academic paper released on Friday."

We've come so far on this open source project, however the class that developed it is now finished. The UI needs a work over, but the giant success is that this is a LIVE version of the Stanfords version. We're super close to finishing something that is super cool.

Check it out at: https://wwpd-ai.vercel.app/

I'm hoping that we can get some help from the community to help bring it to a place where the LLM gives a ted talk on whatever a topic that the user chooses.

Username and pass:
hug@huggingface.com
12345678

raincandy-u DRRRRDFFD osanseviero oceansweep clem abidlabs


Hello everyone, looking to share and learn more most of my experience is security so I will be doing more learning in order to secure AI

adaboost19 raincandy-u nachoyawn sbrandeis


Hey everyone! 👋
This is my first post here and I’m super excited to start with not just one, but two awesome updates! 🚀

Some of you might already know that I recently started my internship at Hugging Face. I’m grateful to be a part of the LLMs evaluation team and the Open LLM Leaderboard! 🤗

First up, we’ve got some big news: we’ve just completed the evaluations for the https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1, and guess what? It’s now the top-performing pretrained model on the Open LLM Leaderboard! A huge shoutout to Mistral! 🏆👏 You can see more details and check out the evaluation results right here – https://huggingface.co/datasets/open-llm-leaderboard/details_mistral-community__Mixtral-8x22B-v0.1

Next, I’m excited to share a cool new feature – you can now search for models on the Open LLM Leaderboard by their licenses! 🕵♂ This feature will help you find the perfect model for your projects way faster. Just type "license: MIT" as a test run!

I'd be super happy if you'd follow me here for more updates on the Leaderboard and other exciting developments. Can’t wait to share more with you soon! ✨

clefourrier julien-c osanseviero santiagokomadina DmitryRyumin victor jkepler81 samusenps sugatoray clem oceansweep


GoMAvatar: Efficient Animatable Human Modeling from Monocular Video
  Using Gaussians-on-Mesh (2404.07991)

🕺🎬🔥 New Research Alert - CVPR 2024 (Avatars Collection)! 🔥🤖⚡
📄 Title: GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh

📝 Description: GoMAvatar is an efficient method for real-time, high-quality, animatable human modeling from a single monocular video. It combines the rendering quality of Gaussian splatting with the geometry modeling capabilities of deformable meshes, enabling realistic digital avatars that can be rearticulated in new poses and rendered from novel angles, while seamlessly integrating with graphics pipelines.

👥 Authors: Jing Wen, Xiaoming Zhao, Zhongzheng Ren, Alexander G. Schwing, Shenlong Wang

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2404.07991

🌐 Github Page: https://wenj.github.io/GoMAvatar/
📁 Repository: https://github.com/wenj/GoMAvatar

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #GoMAvatar #3DAvatar #3DAnimation #AnimatableAvatars #MonocularVideo #RealTimeRendering #HumanModeling #CVPR2024 #DeepLearning #Innovation


Introducing Zephyr 141B-A35B 🪁:

https://huggingface.co/HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1

Yesterday, Mistral released their latest base model (via magnet link of course 😅) and the community quickly converted it to transformers format and pushed it to the Hub: https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1

Early evals of this model looked extremely strong, so we teamed up with Argilla and KAIST AI to cook up a Zephyr recipe with a few new alignment techniques that came out recently:

🧑🍳 Align the base model with Odds Ratio Preference Optimisation (ORPO). This novel algorithm developed by @JW17 and @nlee-208 and @j6mes and does not require an SFT step to achieve high performance and is thus much more computationally efficient than methods like DPO and PPO.

🦫 Use a brand new dataset of 7k high-quality, multi-turn preferences that has been developed by our friends at Argilla. To create this dataset, they took the excellent Capybara SFT dataset from @LDJnr  https://huggingface.co/datasets/LDJnr/Capybara and converted it into a preference dataset by augmenting the final turn with responses from new LLMs that were then ranked by GPT-4.

What we find especially neat about this approach is that training on 7k samples only takes ~1.3h on 4 H100 nodes, yet produces a model that is very strong on chat benchmarks like IFEval and BBH.

Kudos to @alvarobartt @JW17 and @nlee-208 for this very nice and fast-paced collab!

For more details on the paper and dataset, checkout our collection: https://huggingface.co/collections/HuggingFaceH4/zephyr-orpo-6617eba2c5c0e2cc3c151524

alvarobartt mehungrymunchie nlee-208 SilentWraith JW17 samusenps hakunamatata1997 dakshdeephere NouRed victor osanseviero avinash02 fynnkroeger dillfrescott clem philschmid srisree rafaelsandroni not-lain netynet

beta3 clem SilentWraith JW17 mathiasn1 victor osanseviero NickyNicky alvarobartt dillfrescott philschmid not-lain


ChemLLM: A Chemical Large Language Model (2402.06852)

The first part of ChemLLM Training Dataset Opensourced on HuggingFace🤗 Now!

You can use this curated DPO dataset to enhance your own model's performance on Chemistry or Materials tasks.

English version
https://huggingface.co/datasets/AI4Chem/ChemPref-DPO-for-Chemistry-data-en
Chinese Version
https://huggingface.co/datasets/AI4Chem/ChemPref-DPO-for-Chemistry-data-cn

Keep following our [work](https://huggingface.co/AI4Chem) and [paper](https://huggingface.co/papers/2402.06852).

More Data will be available in weeks, All hail to open source community!

samusenps Csplk osanseviero mmhamdy Joseph717171 ucyang


Leave No Context Behind: Efficient Infinite Context Transformers with
  Infini-attention (2404.07143)

Leave No Context Behind

Efficient Infinite Context Transformers with Infini-attention

https://huggingface.co/papers/2404.07143

This work introduces an efficient method to scale Transformer-based Large Language Models (LLMs) to infinitely long inputs with bounded memory and computation. A key component in our proposed approach is a new attention technique dubbed Infini-attention. The Infini-attention incorporates a compressive memory into the vanilla attention mechanism and builds in both masked local attention and long-term linear attention mechanisms in a single Transformer block. We demonstrate the effectiveness of our approach on long-context language modeling benchmarks, 1M sequence length passkey context block retrieval and 500K length book summarization tasks with 1B and 8B LLMs. Our approach introduces minimal bounded memory parameters and enables fast streaming inference for LLMs.

samusenps Ahmedelm osanseviero clem hiyouga victor xpgx1 chen-yingfa


Does Transformer Interpretability Transfer to RNNs? (2404.05971)

Locating and Editing Factual Associations in Mamba (2404.03646)

🔍 Today's pick in Interpretability & Analysis of LMs: x2 edition!

Today's highlighted works aim reproduce findings from Transformer-centric interpretability literature on new RNN-based architectures such as Mamba and RWKV:

https://huggingface.co/papers/2404.05971 by @MrGonao T. Marshall @norabelrose

https://huggingface.co/papers/2404.03646  by @sensharma @datkinson @davidbau

The first paper applies contrastive activation addition, the tuned lens and probing for eliciting latent knowledge in quirky models to Mamba and RWKV LMs, finding these Transformer-specific methods can be applied with slight adaptation to these architectures, obtaining similar results.

The second work applies the ROME method to Mamba, finding weights playing the role of MLPs in encoding factual relations across several Mamba layers, and can be patched to perform model editing. A new SSM-specific technique is also introduced to emulate attention knockout (value zeroing) revealing information flows similar to the ones in Transformers when processing factual statements.

💻 Code: https://github.com/arnab-api/romba

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9


I stopped procrastinating and finally took the time to write the second article of my series of blog posts on SSM: https://huggingface.co/blog/lbourdois/ssm-2022.
In this blog post, I review the history of SSM models released in 2022, with over 14 models discussed in a synthetic format.
They are separated into two parts: "theoretical" (DSS, S4D, GSS, Mega, S5, etc.) and "applications" (Sashimi, ViS4mer, CCNN, etc.).

To understand everything, it's best to have read the introduction to S4 to SSM blog post first: https://huggingface.co/blog/lbourdois/get-on-the-ssm-train.
All the articles in the series are listed in this space: https://huggingface.co/spaces/lbourdois/SSM_blog_posts

Wishing you a good reading :)


Natural language guidance of high-fidelity text-to-speech with synthetic
  annotations (2402.01912)

Yesterday, we released Parler-TTS and Data-Speech, fully open-source reproduction of work from the paper: https://huggingface.co/papers/2402.01912

Parler-TTS is a lightweight text-to-speech (TTS) model that can generate high-quality, natural sounding speech in the style of a given speaker (gender, pitch, speaking style, etc).

https://huggingface.co/collections/parler-tts/parler-tts-fully-open-source-high-quality-tts-models-66164ad285ba03e8ffde214c

Parler-TTS Mini v0.1, is the first iteration Parler-TTS model trained using 10k hours of narrated audiobooks. It generates high-quality speech with features that can be controlled using a simple text prompt (e.g. gender, background noise, speaking rate, pitch and reverberation).

To improve the prosody and naturalness of the speech further, we're scaling up the amount of training data to 50k hours of speech. The v1 release of the model will be trained on this data, as well as inference optimisations, such as flash attention and torch compile.

https://huggingface.co/parler-tts/parler_tts_mini_v0.1

Data-Speech can be used for annotating speech characteristics in a large-scale setting.

https://huggingface.co/collections/parler-tts/open-source-speech-datasets-annotated-using-data-speech-661648ffa0d3d76bfa23d534

This work is both scalable and easily modifiable and will hopefully help the TTS research community explore new ways of conditionning speech synthesis.

All of the datasets, pre-processing, training code and weights are released publicly under permissive license, enabling the community to build on our work and develop their own powerful TTS models.

sanchit-gandhi reach-vb thomwolf DmitryRyumin osanseviero clem Lyte maxugly sroman3 blanchon finefin erkhem-gantulga vedadom microdiploma

reach-vb thomwolf samusenps osanseviero clem maxugly vedadom

reach-vb thomwolf osanseviero clem maxugly vedadom


Writer team had the opportunity to run an eval  for Mixtral-8x22b, results were interesting.

den0620 kur4i osanseviero clem Jason233 wannaphong firstrow ChuxiJ nbroad


Mistral AI recently released a new Mixtral model. It's another Mixture of Experts model with 8 experts, each with 22B parameters. It requires over 200GB of VRAM to run in float16, and over 70GB of VRAM to run in int4. However, individuals have been successful at finetuning it on Apple Silicon laptops using the MLX framework. It features a 64K context window, twice that of their previous models (32K).

The model was released over torrent, a method Mistral has recently often used for their releases. While the license has not been confirmed yet, a moderator on their Discord server yesterday suggested it was Apache 2.0 licensed.

Sources:
• https://twitter.com/_philschmid/status/1778051363554934874
• https://twitter.com/reach_vb/status/1777946948617605384

osanseviero lewiswatson tansutt victor ajibawa-2023 samusenps antiven0m Diegg tcml simonweniger LucienL marz1 clem


VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild (2403.16973)

Diaries of Open Source. Part 14 🤗

🔥CohereForAI releases Command R+, an open 104B model with:
- Tool usage capabilities
- Specialized in RAGs
- Multilingual
It's one of the first models to surpass GPT-4 in the lmsys arena, check it out!
Model: https://hf.co/CohereForAI/c4ai-command-r-plus
Official demo: https://hf.co/spaces/CohereForAI/c4ai-command-r-plus
Quantized: https://hf.co/CohereForAI/c4ai-command-r-plus-4bit

🎉Google releases a new version of their Gemma instruct models, with improved quality, nicer to converse, and a fancier RL algorithm. The model is similar to Llama 2 70B in the Chat Arena!
Models: https://hf.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b
Try it out in HuggingChat https://hf.co/chat/models/google/gemma-1.1-7b-it

🪄VoiceCraft, a speech editing and TTS SOTA open model
Paper: https://hf.co/papers/2403.16973
Model: https://hf.co/pyp1/VoiceCraft

💻Google released CodeGemma, a family of code generation, completion, and chat models
Blog post: https://hf.co/blog/codegemma
Models: https://hf.co/collections/google/codegemma-release-66152ac7b683e2667abdee11
Report: https://storage.googleapis.com/deepmind-media/gemma/codegemma_report.pdf

Misc models:
🦖T-Rex2, a very powerful object detection model for many applications  https://github.com/IDEA-Research/T-Rex
👀 CT-RATE : A 3D dataset paired with text reports https://hf.co/datasets/ibrahimhamamci/CT-RATE
🐙Octopus v2: a Gemma-based model trained for Android API - extremely fast, better than Llama+RAG, great results https://hf.co/NexaAIDev/Octopus-v2

qnguyen3 YaTharThShaRma999 samusenps danielus linoyts simonweniger clem megrisdal ashwinlimaye USM-Valor merve


Context versus Prior Knowledge in Language Models (2404.04633)

🔍 Today's pick in Interpretability & Analysis of LMs: Context versus Prior Knowledge in Language Models by @kdu4108 @vesteinn @niklasstoehr J. C. White A. Schein @rcotterell

This work examines the influence of context versus memorized knowledge in LMs through the lens of the shift caused by contexts at various degrees of informativeness to the models' predictive distribution. Understanding this difference is especially important in the context of knowledge conflicts between memorized and contextual information.

Authors propose disentangling context influence in terms of "persuasion", i.e. how impactful is the inclusion of the context for answers of a given query/entity pair,  and "susceptibility", i.e. how much answers of a given query/entity pair are likely to be swayed by the presence of context, and operationalize these concepts using information-theoretic measures akin to mutual information.

The two metrics are validated using a synthetic dataset sourced from a knowledge graph. Analysis shows that:

-  The degree of persuasiveness of relevant contexts increases with the increase of model size (interesting implications here for the jailbreaking of LLMs!)
- assertive contexts tend to be more persuasive for closed queries (yes/no) and mid-sized models
- Negation affect context persuasiveness
- Familiar entities (explored as real vs. fake, more frequent in training data and more connected in the KG) are less susceptible to context influence

Finally, authors suggest applications of the persuasion/susceptibility framing for social science analyses and gender bias evaluation.

💻 Code: https://github.com/kdu4108/measureLM
📄 Paper: https://huggingface.co/papers/2404.04633

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9


I'm extending the AI Comic Factory (to follow-up on the "bring your own model" philosophy 🤗)  to support the broader LLM ecosystem of our other vendor friends!

Here is the announcement:
https://huggingface.co/spaces/jbilcke-hf/ai-comic-factory/discussions/723

This is an experimental feature, some models/vendors might require parameter tuning but I haven't tested all of them yet (only a bit of GPT-4 👀)

Let me know if you experience any issues!

https://huggingface.co/spaces/jbilcke-hf/ai-comic-factory


DGInStyle: Domain-Generalizable Semantic Segmentation with Image
  Diffusion Models and Stylized Semantic Control (2312.03048)

Another gem from our lab — DGInStyle! We use Stable Diffusion to generate semantic segmentation data for autonomous driving and train domain-generalizable networks.

📟 Website: https://dginstyle.github.io
🧾 Paper: https://arxiv.org/abs/2312.03048
🤗 Hugging Face Paper: https://huggingface.co/papers/2312.03048
🤗 Hugging Face Model: https://huggingface.co/yurujaja/DGInStyle
🐙 Code: https://github.com/yurujaja/DGInStyle

In a nutshell, our pipeline overcomes the resolution loss of Stable Diffusion latent space and the style bias of ControlNet, as shown in the attached figures. This allows us to generate sufficiently high-quality pairs of images and semantic masks to train domain-generalizable semantic segmentation networks.

Team: Yuru Jia (@yurujaja), Lukas Hoyer, Shengyu Huang, Tianfu Wang (@Tianfwang), Luc Van Gool, Konrad Schindler, and Anton Obukhov (@toshas).

samusenps clem Joseph717171 oceansweep Timilla

osanseviero clem Joseph717171 oceansweep


LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders (2404.05961)

LLM2Vec

Large Language Models Are Secretly Powerful Text Encoders

https://huggingface.co/papers/2404.05961

Large decoder-only language models (LLMs) are the state-of-the-art models on most of today's NLP tasks and benchmarks. Yet, the community is only slowly adopting these models for text embedding tasks, which require rich contextualized representations. In this work, we introduce LLM2Vec, a simple unsupervised approach that can transform any decoder-only LLM into a strong text encoder. LLM2Vec consists of three simple steps: 1) enabling bidirectional attention, 2) masked next token prediction, and 3) unsupervised contrastive learning. We demonstrate the effectiveness of LLM2Vec by applying it to 3 popular LLMs ranging from 1.3B to 7B parameters and evaluate the transformed models on English word- and sequence-level tasks. We outperform encoder-only models by a large margin on word-level tasks and reach a new unsupervised state-of-the-art performance on the Massive Text Embeddings Benchmark (MTEB). Moreover, when combining LLM2Vec with supervised contrastive learning, we achieve state-of-the-art performance on MTEB among models that train only on publicly available data. Our strong empirical results and extensive analysis demonstrate that LLMs can be effectively transformed into universal text encoders in a parameter-efficient manner without the need for expensive adaptation or synthetic GPT-4 generated data.

SivilTaram osanseviero CarlLee oliverguhr KvrParaskevi m18coppola celinah


𝙏𝙧𝙮 𝙞𝙩 𝙝𝙚𝙧𝙚 👉

𝙏𝙧𝙮 𝙞𝙩 𝙝𝙚𝙧𝙚 👉

𝗡𝗲𝘄 𝗦𝗽𝗮𝗰𝗲: 𝘼𝙄 𝙏𝙧𝙖𝙫𝙚𝙡 𝙥𝙡𝙖𝙣𝙣𝙚𝙧 🗺🏕  Plan your next vacation in a few minutes!

I wanted to try out if a powerful LLM like Mixtral-8x7b had geographical reasoning capabilities.
So I built a small space that prompts the LLM to provide a JSON list of places based on a user input.

And the result was impressive! 🤯

⇒ 𝗜𝘁 𝘀𝗲𝗲𝗺𝘀 𝗹𝗶𝗸𝗲 𝗠𝗶𝘅𝘁𝗿𝗮𝗹 𝗵𝗮𝘀 𝗮 𝗴𝗿𝗮𝘀𝗽 𝗼𝗳 𝗴𝗲𝗼𝗴𝗿𝗮𝗽𝗵𝗶𝗰𝗮𝗹 𝗰𝗼𝗻𝗰𝗲𝗽𝘁𝘀 𝗹𝗶𝗸𝗲 𝗡𝗼𝗿𝘁𝗵 - 𝗦𝗼𝘂𝘁𝗵, 𝗼𝗿 𝘀𝗽𝗮𝘁𝗶𝗮𝗹 𝗮𝗹𝗶𝗴𝗻𝗺𝗲𝗻𝘁.🧭 Not just describing these concepts, but really applying them in practice, for instance to successfully answer "give me 4 European cities that are aligned on the map". This is a 𝗻𝗶𝗰𝗲 𝗲𝘅𝗮𝗺𝗽𝗹𝗲 𝗼𝗳 𝗮𝗻 𝗲𝗺𝗲𝗿𝗴𝗲𝗻𝘁 𝗰𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝘆, since nothing in the LLM's training data should prepare it for this specific task.

Anyway, I added API calls and a nice visualization on top of the LLM, streaming output, caching for the answers and locations... and ta-da! ✨ I got the 𝗔𝗜 𝗧𝗿𝗮𝘃𝗲𝗹 𝗣𝗹𝗮𝗻𝗻𝗲𝗿.

𝙔𝙤𝙪 𝙘𝙖𝙣 𝙙𝙚𝙨𝙘𝙧𝙞𝙗𝙚 𝙞𝙩 𝙮𝙤𝙪𝙧 𝙩𝙧𝙞𝙥, 𝙖𝙣𝙙 𝙞𝙩 𝙬𝙞𝙡𝙡 𝙘𝙤𝙢𝙚 𝙪𝙥 𝙬𝙞𝙩𝙝 𝙣𝙞𝙘𝙚 𝙖𝙣𝙙 𝙘𝙤𝙣𝙫𝙚𝙣𝙞𝙚𝙣𝙩 𝙡𝙤𝙘𝙖𝙩𝙞𝙤𝙣𝙨!

𝙏𝙧𝙮 𝙞𝙩 𝙝𝙚𝙧𝙚 👉 https://huggingface.co/spaces/m-ric/ai-travel-planner

Thank you @freddyaboulton for the 𝚐𝚛𝚊𝚍𝚒𝚘_𝚏𝚘𝚕𝚒𝚞𝚖 component, and @clem , @pngwn , @abidlabs  for your ideas and support!

pngwn lunarflu osanseviero victor clem monsoon-nlp freddyaboulton

pngwn lunarflu clem Locutusque adamelliotfields raincandy-u freddyaboulton


🎉 Introducing nanoLLaVA, a powerful multimodal AI model that packs the capabilities of a 1B parameter vision language model into just 5GB of VRAM. 🚀 This makes it an ideal choice for edge devices, bringing cutting-edge visual understanding and generation to your devices like never before. 📱💻

Model: https://huggingface.co/qnguyen3/nanoLLaVA 🔍
Spaces: https://huggingface.co/spaces/qnguyen3/nanoLLaVA (thanks to @merve)

Under the hood, nanoLLaVA is based on the powerful https://huggingface.co/vilm/Quyen-SE-v0.1 (my Qwen1.5-0.5B finetune) and Google's impressive https://huggingface.co/google/siglip-so400m-patch14-384. 🧠 The model is trained using a data-centric approach to ensure optimal performance. 📊

In the spirit of transparency and collaboration, all code and model weights are open-sourced under the Apache 2.0 license. 🤝

samusenps SivilTaram DmitryRyumin m-ric osanseviero ArthurFischel den0620 2dts merve qnguyen3 ggcristian

samusenps SivilTaram ndamulelonemakh osanseviero penma 2dts cnmoro merve


PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual
  Observations (2404.04421)

🚀🕺🌟 New Research Alert (Avatars Collection)! 🌟💃🚀
📄 Title: PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations 🔝

📝 Description: PhysAvatar is a novel framework that uses inverse rendering and physics to autonomously reconstruct the shape, appearance, and physical properties of clothed human avatars from multi-view video data.

👥 Authors: Yang Zheng et al.

🔗 Paper: https://huggingface.co/papers/2404.04421

🌐 GitHub Page: https://qingqing-zhao.github.io/PhysAvatar

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #PhysAvatar #DigitalHumans #InverseRendering #PhysicsSimulation #AvatarModeling #ClothSimulation #PhotorealisticRendering #ComputerVision #DeepLearning #Innovation


**Some updates on GLiNER**

🆕 A new commercially permissible multilingual version is available urchade/gliner_multiv2.1

🐛 A subtle bug that causes performance degradation on some models has been corrected. Thanks to @yyDing1 for raising the issue.

```
from gliner import GLiNER

# Initialize GLiNER
model = GLiNER.from_pretrained("urchade/gliner_multiv2.1")

text = "This is a text about Bill Gates and Microsoft."

# Labels for entity prediction
labels = ["person", "organization", "email"]

entities = model.predict_entities(text, labels, threshold=0.5)

for entity in entities:
    print(entity["text"], "=>", entity["label"])
```

tomaarsen osanseviero aari1995 Bailefan erickdp mrdbourke


SwapAnything: Enabling Arbitrary Object Swapping in Personalized Visual
  Editing (2404.05717)

SwapAnything is a new method that allows swapping any object in an image with personalized concepts given by a reference image.

Key points:
1⃣ It uses pre-trained diffusion models to enable precise and high-fidelity object swapping in images.
2⃣Targeted variable swapping ensures perfect background preservation while swapping specific areas.
3⃣SwapAnything achieves good results in single-object, multi-object, partial-object, and cross-domain swapping tasks.

Paper: https://huggingface.co/papers/2404.05717
Project page: https://swap-anything.github.io

Congrats to the authors for their work!


Dense Training, Sparse Inference: Rethinking Training of
  Mixture-of-Experts Language Models (2404.05567)

Thrilled to unveil DS-MoE: a dense training and sparse inference scheme for enhanced computational and memory efficiency in your MoE models! 🚀🚀🚀

Discover more in our blog: https://huggingface.co/blog/bpan/ds-moe and dive into the details with our paper: https://huggingface.co/papers/2404.05567

samusenps umuthopeyildirim zoratrox osanseviero victor ybelkada mayank-mishra


More open research updates 🧵

Performance estimation is currently the best way to quantify the impact of data drift on model performance. 💡

I've been benchmarking performance estimation methods (CBPE and M-CBPE) against data drift signals.

I'm using drift results as features for many regression algorithms, and then I'm taking those to estimate the model's performance. Finally, I'm measuring the Mean Absolute Error (MAE) between the regression models' predictions and actual performance.

So far, for all my experiments, performance estimation methods do better than drift signals. 👨🔬

Bear in mind that these are some early results, I'm running the flow on more datasets as we speak.

Hopefully, by next week, I will have more results to share 👀


Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs (2404.05719)

Ferret-UI

Grounded Mobile UI Understanding with Multimodal LLMs

https://huggingface.co/papers/2404.05719

Recent advancements in multimodal large language models (MLLMs) have been noteworthy, yet, these general-domain MLLMs often fall short in their ability to comprehend and interact effectively with user interface (UI) screens. In this paper, we present Ferret-UI, a new MLLM tailored for enhanced understanding of mobile UI screens, equipped with referring, grounding, and reasoning capabilities. Given that UI screens typically exhibit a more elongated aspect ratio and contain smaller objects of interest (e.g., icons, texts) than natural images, we incorporate "any resolution" on top of Ferret to magnify details and leverage enhanced visual features. Specifically, each screen is divided into 2 sub-images based on the original aspect ratio (i.e., horizontal division for portrait screens and vertical division for landscape screens). Both sub-images are encoded separately before being sent to LLMs. We meticulously gather training samples from an extensive range of elementary UI tasks, such as icon recognition, find text, and widget listing. These samples are formatted for instruction-following with region annotations to facilitate precise referring and grounding. To augment the model's reasoning ability, we further compile a dataset for advanced tasks, including detailed description, perception/interaction conversations, and function inference. After training on the curated datasets, Ferret-UI exhibits outstanding comprehension of UI screens and the capability to execute open-ended instructions. For model evaluation, we establish a comprehensive benchmark encompassing all the aforementioned tasks. Ferret-UI excels not only beyond most open-source UI MLLMs, but also surpasses GPT-4V on all the elementary UI tasks.

samusenps keenyou fscheffczyk KvrParaskevi


Very excited to share the first two official Gemma variants from Google! Today at Google Cloud Next, we announced cutting-edge models for code and research!

First, https://huggingface.co/collections/google/codegemma-release-66152ac7b683e2667abdee11 -  a new set of code-focused Gemma models at 2B and 7B, in both pretrained and instruction-tuned variants. These exhibit outstanding performance on academic benchmarks and (in my experience) real-life usage. Read more in the excellent HuggingFace blog: https://huggingface.co/blog/codegemma

Second, (https://huggingface.co/collections/google/recurrentgemma-release-66152cbdd2d6619cb1665b7a), which is based on the outstanding Google DeepMind research in Griffin: https://arxiv.org/abs/2402.19427. RecurrentGemma is a research variant that enables higher throughput and vastly improved memory usage. We are excited about new architectures, especially in the lightweight Gemma sizes, where innovations like RecurrentGemma can scale modern AI to many more use cases.

For details on the launches of these models, check out our launch blog -- and please do not hesitate to send us feedback. We are excited to see what you build with CodeGemma and RecurrentGemma!

Huge thanks to the Hugging Face team for helping ensure that these models work flawlessly in the Hugging Face ecosystem at launch!

andysalerno kramp rmdhirr manvinder osanseviero reach-vb philschmid eliebak victor nsarrazin BrigitteTousi m-ric tomaarsen lewtun VictorSanh andrewrreed lysandre Wauplin julien-c mcpotato yjernite lunarflu linoyts Tonic pcuenq Sintayew4 icaio

osanseviero reach-vb philschmid eliebak victor nsarrazin BrigitteTousi m-ric tomaarsen lewtun VictorSanh andrewrreed lysandre Wauplin brunatrevelin julien-c yjernite lunarflu Tonic pcuenq Sintayew4 Blane187 clefourrier

osanseviero reach-vb philschmid victor nsarrazin BrigitteTousi tomaarsen lewtun andrewrreed lysandre Wauplin julien-c lunarflu Tonic pcuenq Blane187 icaio


🎉  Super @DeepMount00  just released 𝗚𝗲𝗺𝗺𝗮_𝗤𝗔_𝗜𝗧𝗔_𝘃𝟯 𝗹𝗲𝗮𝗱𝗶𝗻𝗴 the 𝗥𝗔𝗚 𝘁𝗮𝘀𝗸 on the Italian 𝗟𝗟𝗠_𝗜𝗧𝗔_𝗟𝗘𝗔𝗗𝗘𝗥𝗕𝗢𝗔𝗥𝗗. The model is a fine tuned version of Gemma 2B.
Model details: https://huggingface.co/DeepMount00/Gemma_QA_ITA_v3
Explore the full RAG section rankings here: https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard on section Classifica RAG


🔁 AutoMerger created the best 7B model on the Open LLM Leaderboard

By randomly combining top models from the Open LLM Leaderboard, AutoMerger created YamshadowExperiment28-7B. The model is three weeks old and has been at the top of the leaderboard for a week now. It was created through a simple SLERP merge of:

- https://huggingface.co/automerger/YamShadow-7B (another top model created by AutoMerger)
- https://huggingface.co/yam-peleg/Experiment28-7B (a top model from @yam-peleg)

1/ On the Open LLM Leaderboard, it managed to outperform the excellent M7-7b model, which has been the #1 7B model for a while now.

2/ On the YALL leaderboard, YamshadowExperiment28-7B is ranked as the 9th best-performing automerge (but note that the scores are very close to each other). Compared to others, it does not perform particularly well on AGIEval or Bigbench.

3/ Thanks to @sam-paech , I have scores on EQ-Bench, where it managed to outperform all of my previous models. It even surpasses recent models such as DBRX instruct, Qwen1.5 32B Chat, and Cohere's Command R+.

Surprisingly, it does not support ChatML or Mistral Instruct, unlike my other merges (which are part of its family tree). Alpaca works well 99% of the time, but the model can sometimes produce a lot of "INST" tokens for no reason.

In my experiments, YamshadowExperiment28-7B doesn't seem smarter than other successful merges like AlphaMonarch. On the contrary, I found several mathematical or reasoning problems where it fails.

Considering these results, it looks like it might overfit the Open LLM Leaderboard. I guess it's anything but surprising when you randomly merge 156 models.

🤗 Model: https://huggingface.co/automerger/YamshadowExperiment28-7B
🔁 AutoMerger: https://huggingface.co/spaces/mlabonne/AutoMerger

victor AtAndDev Tonic vladbogo rmdhirr samusenps Gargaz osanseviero kgourgou Kukedlc Ramikan-BR

neeravkumar Heng666 Gargaz osanseviero mylesgoose kgourgou eliolio Muhammad2003 Ramikan-BR


Do language models plan ahead for future tokens? (2404.00859)

🔍 Today's pick in Interpretability & Analysis of LMs: Do language models plan ahead for future tokens? by W. Wu @jxm @lionellevine

This work aims to evaluate whether language models exhibit implicit planning during generation.

Authors propose two hypotheses that could result in planning-like behavior:

- Pre-caching: the model engages in computation that is functional to future, but not current, predictions.

- Breadcrumbs: Features contributing to the current prediction happen to also be the ones improving future ones.

To validate which behavior is observed in practice, authors note that off-diagonal gradients for weight matrices across the model are the ones responsible for pre-caching, and craft a variant of gradient descent (myopic descent) to remove such terms from the optimization procedure.

Using a synthetic dataset, authors demonstrate that pre-caching occurs in Transformers language models. However, for natural language settings the LM is observed to leverage breadcrumbs from previous passes even in the case of myopic training, rendering the latter hypothesis more plausible to account for model behavior.

📄 Paper: https://huggingface.co/papers/2404.00859

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9


LegalKit Retrieval, a binary Search with Scalar (int8) Rescoring through French legal codes is now available as a 🤗 Space.

This process is designed to be memory efficient and fast, with the binary index being small enough to fit in memory and the int8 index being loaded as a view. Additionally, the binary index is much faster (up to 32x) to search than the float32 index, while the rescoring is also extremely efficient.

This space also showcases the tsdae-lemone-mbert-base, a sentence embedding model based on BERT fitted using Transformer-based Sequential Denoising Auto-Encoder for unsupervised sentence embedding learning with one objective : french legal domain adaptation.

Link to the 🤗 Space : https://huggingface.co/spaces/louisbrulenaudet/legalkit-retrieval

Notes:
The SentenceTransformer model currently in use is in beta and may not be suitable for direct use in production.

colin-r-carter marksmithh osanseviero tomaarsen samusenps ethanker


Hi HuggingFacers!🤗

As you may have probably heard, in the past weeks three Tech Giants (Microsoft, Amazon and Google) announced that they would bet on nuclear reactors to feed the surging energy demand of data centers, driven by increasing AI data and computational flows.

I try to explain the state of AI energy consumptions, its environmental impact and the key points of "turning AI nuclear" in my last article on HF community blog: https://huggingface.co/blog/as-cle-bert/ai-is-turning-nuclear-a-review

Enjoy the reading!🌱

rmayormartins John6666 den0620 daniel-ltw not-lain louisbrulenaudet SriramRokkam odyss3y julesbsz


Last Thursday at KaggleX organized by Google, I presented a workshop on "Unlocking the Power of Large Language Models (LLMs) for Business Applications" where I explained how we can reduce the size of LLM models to make them more suitable for business use and addressing common resource limitations.
https://drive.google.com/file/d/1p5sT4_DeyBuwCqmYt4dCJKZOgLMpESzR/view


Every adult on the planet knows what a vector is and has the basic understanding of how they are utilized right in their heads. You just don't know it as vector math. You do not know a 2-D vector as a 2-D vector, you know it as a graph. Want to know more? Check out this video, I break down the concept in about 10 minutes and I am positive you will fully understand it by the end: https://youtu.be/Iny2ughcGsA


Artificial Minds, Human Consequences: Unraveling AI’s Impact on Education, Cognition, and Cultural Production

https://empereur-pirate.medium.com/artificial-minds-human-consequences-unraveling-ais-impact-on-education-cognition-and-cultural-a503b88d4524

This text critiques AI language models, particularly in education, arguing their use hinders deep learning, critical thinking, and social-emotional development despite automating tasks. It raises copyright concerns, highlighting lawsuits against companies like OpenAI and Microsoft. AI "creativity" is challenged, emphasizing its reliance on stochastic processes and lack of true understanding. Ethical implications, including bias and misuse, are explored, along with environmental costs. The text contrasts the human brain's complex temporalities and adaptability with the static nature of current AI models, which lack genuine long-term memory and continuous learning.


Made a notable change to the TTS Arena fork. I do not think anyone is interested in which bottomfeeder TTS is better than another beside it. So one of the top 5 TTS is always chosen in a challenge for more scrutiny. Also these top 5 are taken from preliminary results.
https://huggingface.co/spaces/Pendrokar/TTS-Spaces-Arena


SambaNova ☁
⚡ Inference API with cURL Demo: https://huggingface.co/spaces/prithivMLmods/sambanova-inference-api

🔗Sambanova API Documentation : (grab your APIs here)  https://cloud.sambanova.ai/apis

```
export SAMBANOVA_API_KEY=<your token>
```
Sambanova's Inference API.

```
pip install sambanova-gradio
```
SambaNova X Gradio

```
import gradio as gr
import sambanova_gradio

gr.load(
    name='Meta-Llama-3.1-405B-Instruct',
    src=sambanova_gradio.registry,
).launch()
```
📃 Documentation: https://community.sambanova.ai/docs

John6666 victor akhaliq AtAndDev prithivMLmods


I'm now working on finetuning of coding models. If you are GPU-hungry like me, you will find quantized models very helpful. But quantization for finetuning and inference are different and incompatible. So I made two collections here.

Inference (GGUF, via Ollama, CPU is enough)
https://huggingface.co/collections/onekq-ai/ollama-ready-coding-models-67118c3cfa1af2cf04a926d6

Finetuning (Bitsandbytes, QLora, GPU is needed)
https://huggingface.co/collections/onekq-ai/qlora-ready-coding-models-67118771ce001b8f4cf946b2

For quantization, the inference models are far more popular on HF than finetuning models. I use https://huggingface.co/QuantFactory to generate inference models (GGUF), and there are a few other choices.

But there hasn't been such a service for finetuning models. DIY isn't too hard though. I made a few myself and you can find the script in the model cards. If the original model is small enough, you can even do it on a free T4 (available via Google Colab).

If you know a (small) coding model worthy of quantization, please let me know and I'd love to add it to the collections.

djuna John6666 valentimarco RalFinger rajkumar321 regisss


Large Language Models in Targeted Sentiment Analysis (2404.12342)

📢 Excited to share that our studies 📄 "Large Language Models in Targeted Sentiment Analysis for Russian" has recently become in 📘 Springer Lobachevskii Journal of Mathematics 🥳✨ ...

📘 https://link.springer.com/article/10.1134/S1995080224603758

In this studies we provide such a diverse look and experiments over various 🤖 LLM models 🤖 scaled from 7B in two different modes: ❄ zero-shot and 🔥 fine-tuned (Flan-T5 only) using Three-Hop reasoning technique.
We showcase the importance of performing:
💚 text translation into English
💚 application on Chain-of-Thought for Implicit Sentiment Analysis

More:
📄 Arxiv: https://arxiv.org/abs/2404.12342
🧑💻 Code: https://github.com/nicolay-r/Reasoning-for-Sentiment-Analysis-Framework
🤗 Models: https://huggingface.co/papers/2404.12342
🎥 Video @NLPSummit: https://www.youtube.com/watch?v=qawLJsRHzB4

THOR: https://github.com/scofield7419/THOR-ISA