𝐀𝐝𝐝𝐢𝐭𝐢𝐨𝐧𝐚𝐥 𝐢𝐧𝐬𝐢𝐠𝐡𝐭𝐬 💡

𝐀𝐝𝐝𝐢𝐭𝐢𝐨𝐧𝐚𝐥 𝐢𝐧𝐬𝐢𝐠𝐡𝐭𝐬 💡

𝐏𝐚𝐩𝐞𝐫 𝐑𝐞𝐯𝐢𝐞𝐰: 𝐑𝐡𝐨-𝟏 - 𝐃𝐨 𝐧𝐨𝐭 𝐮𝐬𝐞 𝐚𝐥𝐥 𝐭𝐨𝐤𝐞𝐧𝐬 𝐞𝐪𝐮𝐚𝐥𝐥𝐲 𝐢𝐧 𝐲𝐨𝐮𝐫 𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠! ⚖⛔

A new paper topping Daily papers questions a hidden assumption in LLM training:

🤔 𝙎𝙝𝙤𝙪𝙡𝙙 𝙬𝙚 𝙧𝙚𝙖𝙡𝙡𝙮 𝙪𝙨𝙚 𝙖𝙡𝙡 𝙩𝙤𝙠𝙚𝙣𝙨 𝙚𝙦𝙪𝙖𝙡𝙡𝙮 𝙞𝙣 𝙤𝙪𝙧 𝙇𝙇𝙈'𝙨 𝙩𝙧𝙖𝙞𝙣𝙞𝙣𝙜 ?

Some tokens are more relevant than others, and some are mostly noise (just look up the history of 𝘚𝘰𝘭𝘪𝘥𝘎𝘰𝘭𝘥𝘔𝘢𝘨𝘪𝘬𝘢𝘳𝘱).

So this paper introduces 𝗦𝗲𝗹𝗲𝗰𝘁𝗶𝘃𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝗶𝗻𝗴, which is actually really simple:
➡ A specific metric measures the relevance of each token. Then during training, only the top k% tokens for this relevance metric count in the loss calculation.

Authors test this method by training models on the difficult MATH dataset (only competition mathematics problems).

➡ Their technique seems like a new must-do in LLM training: Training is much faster and reaches an impressive performance!

𝐑𝐞𝐬𝐮𝐥𝐭𝐬:
◆ ⏱ Training is x5 to x10 faster to reach equivalent performance compared to standard language modeling.
◆ 💪 Their 1B model achieves close to GPT4 Chain-of-Thought performance on MATH!
◆ 🚀 Their 7B model match performance of the state-of-the-art DeepSeek for the same size, while trained on only 3% of tokens

𝐀𝐝𝐝𝐢𝐭𝐢𝐨𝐧𝐚𝐥 𝐢𝐧𝐬𝐢𝐠𝐡𝐭𝐬 💡
◆ Datasets used for pre-training, even after pre-filtering, still contain a large proportion of noisy tokens 😖
◆ Authors show that when you reduce loss on noisy tokens, you actually reduce accuracy (Figure 7). So Selective Language Modeling seems fundamental! ✅

Find great reads in @akhaliq 's Daily Papers 👉 https://huggingface.co/papers
Paper added to my collection 👉 https://huggingface.co/collections/m-ric/spinning-up-in-llms-659e698f9dd5a71bd3f579a7


We just released a new MoE model (meraGPT/mera-mix-4x7B) that is half as large as Mixtral-8x7B while still been competitive with it across different benchmarks. mera-mix-4x7B achieves 76.37 on the open LLM eval.

You can check mera-mix-4x7B out on HF here - https://huggingface.co/meraGPT/mera-mix-4x7B


ORPO: Monolithic Preference Optimization without Reference Model (2403.07691)

Could more DPO-style preference data be crucial for enhancing open LLMs across different languages?

Leveraging a 7k preference dataset Argilla (@alvarobartt), Hugging Face (@lewtun) and Kaist AI (@JW17 & @nlee-208)
utilized Kaist AI's recently introduced ORPO technique https://huggingface.co/papers/2403.07691 with the latest MistralAI MOE model to create a very high-performing open LLM: https://huggingface.co/HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1

Since ORPO doesn't require a separate SFT stage, all that is needed is a strong base model + high-quality DPO-style datasets.

Currently, there is a significant lack of non-English DPO datasets. Filling this gap could significantly improve open LLMs in various languages.

You can get an overview of the current state of DPO datasets across different languages here: https://huggingface.co/spaces/DIBT/preference_data_by_language

JW17 alvarobartt osanseviero nlee-208 NMontanaBrown Violette Heng666 raincandy-u DataSoul


On Agentic AI: Autonomy Is All You Need!

There is this remarkable beauty in witnessing an AI system autonomously completes complex tasks with a level of brilliance that supersedes our reasoning capabilities/expectations - It is the holy grail of creation.

Giving your AI agents autonomy is analogous to us having "free will" and everything else thereafter is a cascade of possibilities and potentials waiting to unfold.

As brilliantly said by @AndrewNg  "It’s a beautiful thing when you see an agent autonomously decide to do things in ways that you had not anticipated, and succeed as a result!"

Autonomy in Agentic AI
 - augments agents' decision-making capabilities.
 - enables adaptation to diverse environments.
 - facilitates real-time learning and improvement.
 - fosters dynamic multi-agent collaboration.
 - promotes efficient and independent task execution.
 - drives innovation in dynamic and unpredictable
    scenarios.

This what the AutoAgents paper conveyed - A fully autonomous agentic framework that basically gives agents the free will for authentic/compelling creativity.

With just three dynamically predefined agents [ Planner, Agent Observer and Plan Observer ] acting collaboratively they are able to generically create autonomous agents:

```
Agent (A) =  {
     Prompt (P) - defines agent's identity fully ,
     Description (D) - adds specific role identity,
     Toolset (T) - equips the agent with tools,
     Suggestion (S) - offers task execution tips
 }
```

Demo: https://huggingface.co/spaces/LinkSoul/AutoAgents
Code: https://github.com/Link-AGI/AutoAgents
Paper: https://arxiv.org/abs/2309.17288


Fun dataset added last week by  @esind from https://huggingface.co/Anthropic to compare persuasiveness between AI and human outputs:
https://huggingface.co/datasets/Anthropic/persuasion

Jaward kgourgou osanseviero not-lain KvrParaskevi InferenceIllusionist nogully


We're introducing experimental support for `device_map` in Diffusers 🤗

If you have multiple GPUs you want to use to distribute the pipeline models, you can do so. Additionally, this becomes more useful when you have multiple low-VRAM GPUs.

Documentation:
https://huggingface.co/docs/diffusers/main/en/training/distributed_inference#device-placement

🚨 Currently, only "balanced" device mapping strategy is supported.

ajibawa-2023 clem osanseviero Bruno Paolo16 alvarobartt ingTikna


Watch the full tutorial here : https://youtu.be/0t5l6CP9eBg

The tutorial is over 2 hours literally with manually fixed captions and perfect video chapters.

Most Awaited Full Fine Tuning (with DreamBooth effect) Tutorial Generated Images - Full Workflow Shared In The Comments - NO Paywall This Time - Explained OneTrainer - Cumulative Experience of 16 Months Stable Diffusion

In this tutorial, I am going to show you how to install OneTrainer from scratch on your computer and do a Stable Diffusion SDXL (Full Fine-Tuning 10.3 GB VRAM) and SD 1.5 (Full Fine-Tuning 7GB VRAM) based models training on your computer and also do the same training on a very cheap cloud machine from MassedCompute if you don't have such computer.

Tutorial Readme File ⤵
https://github.com/FurkanGozukara/Stable-Diffusion/blob/main/Tutorials/OneTrainer-Master-SD-1_5-SDXL-Windows-Cloud-Tutorial.md

Register Massed Compute From Below Link (could be necessary to use our Special Coupon for A6000 GPU for 31 cents per hour) ⤵
https://bit.ly/Furkan-Gözükara

Coupon Code for A6000 GPU is : SECourses

0:00 Introduction to Zero-to-Hero Stable Diffusion (SD) Fine-Tuning with OneTrainer (OT) tutorial
3:54 Intro to instructions GitHub readme
4:32 How to register Massed Compute (MC) and start virtual machine (VM)
5:48 Which template to choose on MC
6:36 How to apply MC coupon
8:41 How to install OT on your computer to train
9:15 How to verify your Python, Git, FFmpeg and Git installation
12:00 How to install ThinLinc and start using your MC VM
12:26 How to setup folder synchronization and file sharing between your computer and MC VM
13:56 End existing session in ThinClient
14:06 How to turn off MC VM
14:24 How to connect and start using VM
14:41 When use end existing session
16:38 How to download very best OT preset training configuration for SD 1.5 & SDXL models
18:00 How to load configuration preset
18:38 Full explanation of OT configuration and best hyper parameters for SDXL
.
.
.

rafalwiatrowski mehdicode123 samusenps kadirnar HeavyLvy clem Aakashk Matix200 AlexSonar levulinh


Here's a utility component for integrating your Gradio app with Hugging Face. This custom component enables you to search for models, spaces, datasets, and users.

`pip install gradio_huggingfacehub_search`

You can see it in action here. https://huggingface.co/spaces/arcee-ai/mergekit-config-generator

And learn how to use it here https://huggingface.co/spaces/radames/gradio_huggingfacehub_search

samusenps osanseviero srisree clem Gevennou multimodalart


"How expensive is it actually to teach a #LanguageModel German through #finetuning 💰💰💰? We get asked this quite often.

There is no one-size-fits-all answer to this question, as among other factors:
⏹ each fine-tuning is different,
⏹ the hardware used can be a major cost driver,
⏹ the amount and type of training data can extend the process,
⏹ and the skills to be trained can increase the difficulty of fine-tuning.

However, we have broken down the costs incurred for our latest fine-tune ( https://huggingface.co/VAGOsolutions/SauerkrautLM-Qwen-32b)

Base model: https://huggingface.co/Qwen/Qwen1.5-32B
Fine-Tuning Goal: Train German language
Training dataset size: 160,000 SFT data / 110,000 DPO data
Training duration: 72.5 hours (2 epochs SFT / 1 epoch DPO)
GPU: 2x A100 SXM
New model: https://huggingface.co/VAGOsolutions/SauerkrautLM-Qwen-32b

Total cost: 312 euros 💶

These are quite reasonable training costs considering the model now speaks passable German (previously very broken). Depending on the use case and process requirements, this can even be a real alternative to the costly continuous pre-training of foreign language models.

jed-tiotuico osanseviero clem victor fpreiss philipp-zettl joennlae Kearm


Yesterday, xAI announced Grok-1.5 Vision - https://x.ai/blog/grok-1.5v. But more importantly, they also released a new VLM benchmark dataset - RealWorldQA. The only problem was that they released it as a ZIP archive. I fixed that! Now you can use it in your evaluations as a regular HF dataset: https://huggingface.co/datasets/visheratin/realworldqa

musfiqdehan fynnkroeger osanseviero Bruno ajibawa-2023 VictorSanh clem not-lain ShihHong alielfilali01 dinhanhx


Introducing MusicGen Web: AI-powered music generation directly in your browser, built with 🤗 Transformers.js! 🎵

Everything runs 100% locally, meaning there are no calls to an API! 🤯 Since it's served as a static HF space, it costs $0 to host and run! 🔥

We also added the ability to share your generated music to the discussion tab, so give it a try! 👇
https://huggingface.co/spaces/Xenova/musicgen-web

DmitryRyumin adamelliotfields samusenps andrewmilesmurphy1992 Lyte rocca Sword666 ajibawa-2023 ashwinlimaye Retr0REG cdosoftei Cksampa osanseviero clem AlekseiPravdin victor ylacombe alt2023 DataSoul b4s36t4 Noomam nekomeowww

luoweiqiang9527 cdosoftei raincandy-u Cksampa osanseviero clem victor Kce niwram69


Spent the morning designing a super important product: SMALL LANGUAGE MODEL BABY ONESIE https://www.shop.hackernoon.com/all-merch/p/small-language-model-baby-onesie

samusenps visheratin travisking Violette


Hi everyone! I lead a team of students to create a game board community of LLMs that interact to help one of the LLMs create a presentation on an idea.
https://github.com/GooseCube/WWPD.ai
This was based on Standfords unusable initial version:

"A group of researchers at Stanford University and Google have created a miniature RPG-style virtual world similar to The Sims, where 25 characters, controlled by ChatGPT and custom code, live out their lives independently with a high degree of realistic behavior. They wrote about their experiment in a preprint academic paper released on Friday."

We've come so far on this open source project, however the class that developed it is now finished. The UI needs a work over, but the giant success is that this is a LIVE version of the Stanfords version. We're super close to finishing something that is super cool.

Check it out at: https://wwpd-ai.vercel.app/

I'm hoping that we can get some help from the community to help bring it to a place where the LLM gives a ted talk on whatever a topic that the user chooses.

Username and pass:
hug@huggingface.com
12345678

raincandy-u DRRRRDFFD osanseviero oceansweep clem abidlabs


Hello everyone, looking to share and learn more most of my experience is security so I will be doing more learning in order to secure AI

adaboost19 raincandy-u nachoyawn sbrandeis


Hey everyone! 👋
This is my first post here and I’m super excited to start with not just one, but two awesome updates! 🚀

Some of you might already know that I recently started my internship at Hugging Face. I’m grateful to be a part of the LLMs evaluation team and the Open LLM Leaderboard! 🤗

First up, we’ve got some big news: we’ve just completed the evaluations for the https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1, and guess what? It’s now the top-performing pretrained model on the Open LLM Leaderboard! A huge shoutout to Mistral! 🏆👏 You can see more details and check out the evaluation results right here – https://huggingface.co/datasets/open-llm-leaderboard/details_mistral-community__Mixtral-8x22B-v0.1

Next, I’m excited to share a cool new feature – you can now search for models on the Open LLM Leaderboard by their licenses! 🕵♂ This feature will help you find the perfect model for your projects way faster. Just type "license: MIT" as a test run!

I'd be super happy if you'd follow me here for more updates on the Leaderboard and other exciting developments. Can’t wait to share more with you soon! ✨

clefourrier julien-c osanseviero santiagokomadina DmitryRyumin victor jkepler81 samusenps sugatoray clem oceansweep


GoMAvatar: Efficient Animatable Human Modeling from Monocular Video
  Using Gaussians-on-Mesh (2404.07991)

🕺🎬🔥 New Research Alert - CVPR 2024 (Avatars Collection)! 🔥🤖⚡
📄 Title: GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh

📝 Description: GoMAvatar is an efficient method for real-time, high-quality, animatable human modeling from a single monocular video. It combines the rendering quality of Gaussian splatting with the geometry modeling capabilities of deformable meshes, enabling realistic digital avatars that can be rearticulated in new poses and rendered from novel angles, while seamlessly integrating with graphics pipelines.

👥 Authors: Jing Wen, Xiaoming Zhao, Zhongzheng Ren, Alexander G. Schwing, Shenlong Wang

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2404.07991

🌐 Github Page: https://wenj.github.io/GoMAvatar/
📁 Repository: https://github.com/wenj/GoMAvatar

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #GoMAvatar #3DAvatar #3DAnimation #AnimatableAvatars #MonocularVideo #RealTimeRendering #HumanModeling #CVPR2024 #DeepLearning #Innovation


Introducing Zephyr 141B-A35B 🪁:

https://huggingface.co/HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1

Yesterday, Mistral released their latest base model (via magnet link of course 😅) and the community quickly converted it to transformers format and pushed it to the Hub: https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1

Early evals of this model looked extremely strong, so we teamed up with Argilla and KAIST AI to cook up a Zephyr recipe with a few new alignment techniques that came out recently:

🧑🍳 Align the base model with Odds Ratio Preference Optimisation (ORPO). This novel algorithm developed by @JW17 and @nlee-208 and @j6mes and does not require an SFT step to achieve high performance and is thus much more computationally efficient than methods like DPO and PPO.

🦫 Use a brand new dataset of 7k high-quality, multi-turn preferences that has been developed by our friends at Argilla. To create this dataset, they took the excellent Capybara SFT dataset from @LDJnr  https://huggingface.co/datasets/LDJnr/Capybara and converted it into a preference dataset by augmenting the final turn with responses from new LLMs that were then ranked by GPT-4.

What we find especially neat about this approach is that training on 7k samples only takes ~1.3h on 4 H100 nodes, yet produces a model that is very strong on chat benchmarks like IFEval and BBH.

Kudos to @alvarobartt @JW17 and @nlee-208 for this very nice and fast-paced collab!

For more details on the paper and dataset, checkout our collection: https://huggingface.co/collections/HuggingFaceH4/zephyr-orpo-6617eba2c5c0e2cc3c151524

alvarobartt mehungrymunchie nlee-208 SilentWraith JW17 samusenps hakunamatata1997 dakshdeephere NouRed victor osanseviero avinash02 fynnkroeger dillfrescott clem philschmid srisree rafaelsandroni not-lain netynet

beta3 clem SilentWraith JW17 mathiasn1 victor osanseviero NickyNicky alvarobartt dillfrescott philschmid not-lain


ChemLLM: A Chemical Large Language Model (2402.06852)

The first part of ChemLLM Training Dataset Opensourced on HuggingFace🤗 Now!

You can use this curated DPO dataset to enhance your own model's performance on Chemistry or Materials tasks.

English version
https://huggingface.co/datasets/AI4Chem/ChemPref-DPO-for-Chemistry-data-en
Chinese Version
https://huggingface.co/datasets/AI4Chem/ChemPref-DPO-for-Chemistry-data-cn

Keep following our [work](https://huggingface.co/AI4Chem) and [paper](https://huggingface.co/papers/2402.06852).

More Data will be available in weeks, All hail to open source community!

samusenps Csplk osanseviero mmhamdy Joseph717171 ucyang


Leave No Context Behind: Efficient Infinite Context Transformers with
  Infini-attention (2404.07143)

Leave No Context Behind

Efficient Infinite Context Transformers with Infini-attention

https://huggingface.co/papers/2404.07143

This work introduces an efficient method to scale Transformer-based Large Language Models (LLMs) to infinitely long inputs with bounded memory and computation. A key component in our proposed approach is a new attention technique dubbed Infini-attention. The Infini-attention incorporates a compressive memory into the vanilla attention mechanism and builds in both masked local attention and long-term linear attention mechanisms in a single Transformer block. We demonstrate the effectiveness of our approach on long-context language modeling benchmarks, 1M sequence length passkey context block retrieval and 500K length book summarization tasks with 1B and 8B LLMs. Our approach introduces minimal bounded memory parameters and enables fast streaming inference for LLMs.

samusenps Ahmedelm osanseviero clem hiyouga victor xpgx1 chen-yingfa


Does Transformer Interpretability Transfer to RNNs? (2404.05971)

Locating and Editing Factual Associations in Mamba (2404.03646)

🔍 Today's pick in Interpretability & Analysis of LMs: x2 edition!

Today's highlighted works aim reproduce findings from Transformer-centric interpretability literature on new RNN-based architectures such as Mamba and RWKV:

https://huggingface.co/papers/2404.05971 by @MrGonao T. Marshall @norabelrose

https://huggingface.co/papers/2404.03646  by @sensharma @datkinson @davidbau

The first paper applies contrastive activation addition, the tuned lens and probing for eliciting latent knowledge in quirky models to Mamba and RWKV LMs, finding these Transformer-specific methods can be applied with slight adaptation to these architectures, obtaining similar results.

The second work applies the ROME method to Mamba, finding weights playing the role of MLPs in encoding factual relations across several Mamba layers, and can be patched to perform model editing. A new SSM-specific technique is also introduced to emulate attention knockout (value zeroing) revealing information flows similar to the ones in Transformers when processing factual statements.

💻 Code: https://github.com/arnab-api/romba

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9


I stopped procrastinating and finally took the time to write the second article of my series of blog posts on SSM: https://huggingface.co/blog/lbourdois/ssm-2022.
In this blog post, I review the history of SSM models released in 2022, with over 14 models discussed in a synthetic format.
They are separated into two parts: "theoretical" (DSS, S4D, GSS, Mega, S5, etc.) and "applications" (Sashimi, ViS4mer, CCNN, etc.).

To understand everything, it's best to have read the introduction to S4 to SSM blog post first: https://huggingface.co/blog/lbourdois/get-on-the-ssm-train.
All the articles in the series are listed in this space: https://huggingface.co/spaces/lbourdois/SSM_blog_posts

Wishing you a good reading :)


Natural language guidance of high-fidelity text-to-speech with synthetic
  annotations (2402.01912)

Yesterday, we released Parler-TTS and Data-Speech, fully open-source reproduction of work from the paper: https://huggingface.co/papers/2402.01912

Parler-TTS is a lightweight text-to-speech (TTS) model that can generate high-quality, natural sounding speech in the style of a given speaker (gender, pitch, speaking style, etc).

https://huggingface.co/collections/parler-tts/parler-tts-fully-open-source-high-quality-tts-models-66164ad285ba03e8ffde214c

Parler-TTS Mini v0.1, is the first iteration Parler-TTS model trained using 10k hours of narrated audiobooks. It generates high-quality speech with features that can be controlled using a simple text prompt (e.g. gender, background noise, speaking rate, pitch and reverberation).

To improve the prosody and naturalness of the speech further, we're scaling up the amount of training data to 50k hours of speech. The v1 release of the model will be trained on this data, as well as inference optimisations, such as flash attention and torch compile.

https://huggingface.co/parler-tts/parler_tts_mini_v0.1

Data-Speech can be used for annotating speech characteristics in a large-scale setting.

https://huggingface.co/collections/parler-tts/open-source-speech-datasets-annotated-using-data-speech-661648ffa0d3d76bfa23d534

This work is both scalable and easily modifiable and will hopefully help the TTS research community explore new ways of conditionning speech synthesis.

All of the datasets, pre-processing, training code and weights are released publicly under permissive license, enabling the community to build on our work and develop their own powerful TTS models.

sanchit-gandhi reach-vb thomwolf DmitryRyumin osanseviero clem Lyte maxugly sroman3 blanchon finefin erkhem-gantulga vedadom microdiploma

reach-vb thomwolf samusenps osanseviero clem maxugly vedadom

reach-vb thomwolf osanseviero clem maxugly vedadom


Writer team had the opportunity to run an eval  for Mixtral-8x22b, results were interesting.

den0620 kur4i osanseviero clem Jason233 wannaphong firstrow ChuxiJ nbroad


Mistral AI recently released a new Mixtral model. It's another Mixture of Experts model with 8 experts, each with 22B parameters. It requires over 200GB of VRAM to run in float16, and over 70GB of VRAM to run in int4. However, individuals have been successful at finetuning it on Apple Silicon laptops using the MLX framework. It features a 64K context window, twice that of their previous models (32K).

The model was released over torrent, a method Mistral has recently often used for their releases. While the license has not been confirmed yet, a moderator on their Discord server yesterday suggested it was Apache 2.0 licensed.

Sources:
• https://twitter.com/_philschmid/status/1778051363554934874
• https://twitter.com/reach_vb/status/1777946948617605384

osanseviero lewiswatson tansutt victor ajibawa-2023 samusenps antiven0m Diegg tcml simonweniger LucienL marz1 clem


VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild (2403.16973)

Diaries of Open Source. Part 14 🤗

🔥CohereForAI releases Command R+, an open 104B model with:
- Tool usage capabilities
- Specialized in RAGs
- Multilingual
It's one of the first models to surpass GPT-4 in the lmsys arena, check it out!
Model: https://hf.co/CohereForAI/c4ai-command-r-plus
Official demo: https://hf.co/spaces/CohereForAI/c4ai-command-r-plus
Quantized: https://hf.co/CohereForAI/c4ai-command-r-plus-4bit

🎉Google releases a new version of their Gemma instruct models, with improved quality, nicer to converse, and a fancier RL algorithm. The model is similar to Llama 2 70B in the Chat Arena!
Models: https://hf.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b
Try it out in HuggingChat https://hf.co/chat/models/google/gemma-1.1-7b-it

🪄VoiceCraft, a speech editing and TTS SOTA open model
Paper: https://hf.co/papers/2403.16973
Model: https://hf.co/pyp1/VoiceCraft

💻Google released CodeGemma, a family of code generation, completion, and chat models
Blog post: https://hf.co/blog/codegemma
Models: https://hf.co/collections/google/codegemma-release-66152ac7b683e2667abdee11
Report: https://storage.googleapis.com/deepmind-media/gemma/codegemma_report.pdf

Misc models:
🦖T-Rex2, a very powerful object detection model for many applications  https://github.com/IDEA-Research/T-Rex
👀 CT-RATE : A 3D dataset paired with text reports https://hf.co/datasets/ibrahimhamamci/CT-RATE
🐙Octopus v2: a Gemma-based model trained for Android API - extremely fast, better than Llama+RAG, great results https://hf.co/NexaAIDev/Octopus-v2

qnguyen3 YaTharThShaRma999 samusenps danielus linoyts simonweniger clem megrisdal ashwinlimaye USM-Valor merve


Context versus Prior Knowledge in Language Models (2404.04633)

🔍 Today's pick in Interpretability & Analysis of LMs: Context versus Prior Knowledge in Language Models by @kdu4108 @vesteinn @niklasstoehr J. C. White A. Schein @rcotterell

This work examines the influence of context versus memorized knowledge in LMs through the lens of the shift caused by contexts at various degrees of informativeness to the models' predictive distribution. Understanding this difference is especially important in the context of knowledge conflicts between memorized and contextual information.

Authors propose disentangling context influence in terms of "persuasion", i.e. how impactful is the inclusion of the context for answers of a given query/entity pair,  and "susceptibility", i.e. how much answers of a given query/entity pair are likely to be swayed by the presence of context, and operationalize these concepts using information-theoretic measures akin to mutual information.

The two metrics are validated using a synthetic dataset sourced from a knowledge graph. Analysis shows that:

-  The degree of persuasiveness of relevant contexts increases with the increase of model size (interesting implications here for the jailbreaking of LLMs!)
- assertive contexts tend to be more persuasive for closed queries (yes/no) and mid-sized models
- Negation affect context persuasiveness
- Familiar entities (explored as real vs. fake, more frequent in training data and more connected in the KG) are less susceptible to context influence

Finally, authors suggest applications of the persuasion/susceptibility framing for social science analyses and gender bias evaluation.

💻 Code: https://github.com/kdu4108/measureLM
📄 Paper: https://huggingface.co/papers/2404.04633

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9


I'm extending the AI Comic Factory (to follow-up on the "bring your own model" philosophy 🤗)  to support the broader LLM ecosystem of our other vendor friends!

Here is the announcement:
https://huggingface.co/spaces/jbilcke-hf/ai-comic-factory/discussions/723

This is an experimental feature, some models/vendors might require parameter tuning but I haven't tested all of them yet (only a bit of GPT-4 👀)

Let me know if you experience any issues!

https://huggingface.co/spaces/jbilcke-hf/ai-comic-factory


DGInStyle: Domain-Generalizable Semantic Segmentation with Image
  Diffusion Models and Stylized Semantic Control (2312.03048)

Another gem from our lab — DGInStyle! We use Stable Diffusion to generate semantic segmentation data for autonomous driving and train domain-generalizable networks.

📟 Website: https://dginstyle.github.io
🧾 Paper: https://arxiv.org/abs/2312.03048
🤗 Hugging Face Paper: https://huggingface.co/papers/2312.03048
🤗 Hugging Face Model: https://huggingface.co/yurujaja/DGInStyle
🐙 Code: https://github.com/yurujaja/DGInStyle

In a nutshell, our pipeline overcomes the resolution loss of Stable Diffusion latent space and the style bias of ControlNet, as shown in the attached figures. This allows us to generate sufficiently high-quality pairs of images and semantic masks to train domain-generalizable semantic segmentation networks.

Team: Yuru Jia (@yurujaja), Lukas Hoyer, Shengyu Huang, Tianfu Wang (@Tianfwang), Luc Van Gool, Konrad Schindler, and Anton Obukhov (@toshas).

samusenps clem Joseph717171 oceansweep Timilla

osanseviero clem Joseph717171 oceansweep


LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders (2404.05961)

LLM2Vec

Large Language Models Are Secretly Powerful Text Encoders

https://huggingface.co/papers/2404.05961

Large decoder-only language models (LLMs) are the state-of-the-art models on most of today's NLP tasks and benchmarks. Yet, the community is only slowly adopting these models for text embedding tasks, which require rich contextualized representations. In this work, we introduce LLM2Vec, a simple unsupervised approach that can transform any decoder-only LLM into a strong text encoder. LLM2Vec consists of three simple steps: 1) enabling bidirectional attention, 2) masked next token prediction, and 3) unsupervised contrastive learning. We demonstrate the effectiveness of LLM2Vec by applying it to 3 popular LLMs ranging from 1.3B to 7B parameters and evaluate the transformed models on English word- and sequence-level tasks. We outperform encoder-only models by a large margin on word-level tasks and reach a new unsupervised state-of-the-art performance on the Massive Text Embeddings Benchmark (MTEB). Moreover, when combining LLM2Vec with supervised contrastive learning, we achieve state-of-the-art performance on MTEB among models that train only on publicly available data. Our strong empirical results and extensive analysis demonstrate that LLMs can be effectively transformed into universal text encoders in a parameter-efficient manner without the need for expensive adaptation or synthetic GPT-4 generated data.

SivilTaram osanseviero CarlLee oliverguhr KvrParaskevi m18coppola celinah


𝙏𝙧𝙮 𝙞𝙩 𝙝𝙚𝙧𝙚 👉

𝙏𝙧𝙮 𝙞𝙩 𝙝𝙚𝙧𝙚 👉

𝗡𝗲𝘄 𝗦𝗽𝗮𝗰𝗲: 𝘼𝙄 𝙏𝙧𝙖𝙫𝙚𝙡 𝙥𝙡𝙖𝙣𝙣𝙚𝙧 🗺🏕  Plan your next vacation in a few minutes!

I wanted to try out if a powerful LLM like Mixtral-8x7b had geographical reasoning capabilities.
So I built a small space that prompts the LLM to provide a JSON list of places based on a user input.

And the result was impressive! 🤯

⇒ 𝗜𝘁 𝘀𝗲𝗲𝗺𝘀 𝗹𝗶𝗸𝗲 𝗠𝗶𝘅𝘁𝗿𝗮𝗹 𝗵𝗮𝘀 𝗮 𝗴𝗿𝗮𝘀𝗽 𝗼𝗳 𝗴𝗲𝗼𝗴𝗿𝗮𝗽𝗵𝗶𝗰𝗮𝗹 𝗰𝗼𝗻𝗰𝗲𝗽𝘁𝘀 𝗹𝗶𝗸𝗲 𝗡𝗼𝗿𝘁𝗵 - 𝗦𝗼𝘂𝘁𝗵, 𝗼𝗿 𝘀𝗽𝗮𝘁𝗶𝗮𝗹 𝗮𝗹𝗶𝗴𝗻𝗺𝗲𝗻𝘁.🧭 Not just describing these concepts, but really applying them in practice, for instance to successfully answer "give me 4 European cities that are aligned on the map". This is a 𝗻𝗶𝗰𝗲 𝗲𝘅𝗮𝗺𝗽𝗹𝗲 𝗼𝗳 𝗮𝗻 𝗲𝗺𝗲𝗿𝗴𝗲𝗻𝘁 𝗰𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝘆, since nothing in the LLM's training data should prepare it for this specific task.

Anyway, I added API calls and a nice visualization on top of the LLM, streaming output, caching for the answers and locations... and ta-da! ✨ I got the 𝗔𝗜 𝗧𝗿𝗮𝘃𝗲𝗹 𝗣𝗹𝗮𝗻𝗻𝗲𝗿.

𝙔𝙤𝙪 𝙘𝙖𝙣 𝙙𝙚𝙨𝙘𝙧𝙞𝙗𝙚 𝙞𝙩 𝙮𝙤𝙪𝙧 𝙩𝙧𝙞𝙥, 𝙖𝙣𝙙 𝙞𝙩 𝙬𝙞𝙡𝙡 𝙘𝙤𝙢𝙚 𝙪𝙥 𝙬𝙞𝙩𝙝 𝙣𝙞𝙘𝙚 𝙖𝙣𝙙 𝙘𝙤𝙣𝙫𝙚𝙣𝙞𝙚𝙣𝙩 𝙡𝙤𝙘𝙖𝙩𝙞𝙤𝙣𝙨!

𝙏𝙧𝙮 𝙞𝙩 𝙝𝙚𝙧𝙚 👉 https://huggingface.co/spaces/m-ric/ai-travel-planner

Thank you @freddyaboulton for the 𝚐𝚛𝚊𝚍𝚒𝚘_𝚏𝚘𝚕𝚒𝚞𝚖 component, and @clem , @pngwn , @abidlabs  for your ideas and support!

pngwn lunarflu osanseviero victor clem monsoon-nlp freddyaboulton

pngwn lunarflu clem Locutusque adamelliotfields raincandy-u freddyaboulton


🎉 Introducing nanoLLaVA, a powerful multimodal AI model that packs the capabilities of a 1B parameter vision language model into just 5GB of VRAM. 🚀 This makes it an ideal choice for edge devices, bringing cutting-edge visual understanding and generation to your devices like never before. 📱💻

Model: https://huggingface.co/qnguyen3/nanoLLaVA 🔍
Spaces: https://huggingface.co/spaces/qnguyen3/nanoLLaVA (thanks to @merve)

Under the hood, nanoLLaVA is based on the powerful https://huggingface.co/vilm/Quyen-SE-v0.1 (my Qwen1.5-0.5B finetune) and Google's impressive https://huggingface.co/google/siglip-so400m-patch14-384. 🧠 The model is trained using a data-centric approach to ensure optimal performance. 📊

In the spirit of transparency and collaboration, all code and model weights are open-sourced under the Apache 2.0 license. 🤝

samusenps SivilTaram DmitryRyumin m-ric osanseviero ArthurFischel den0620 2dts merve qnguyen3 ggcristian

samusenps SivilTaram ndamulelonemakh osanseviero penma 2dts cnmoro merve


PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual
  Observations (2404.04421)

🚀🕺🌟 New Research Alert (Avatars Collection)! 🌟💃🚀
📄 Title: PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations 🔝

📝 Description: PhysAvatar is a novel framework that uses inverse rendering and physics to autonomously reconstruct the shape, appearance, and physical properties of clothed human avatars from multi-view video data.

👥 Authors: Yang Zheng et al.

🔗 Paper: https://huggingface.co/papers/2404.04421

🌐 GitHub Page: https://qingqing-zhao.github.io/PhysAvatar

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #PhysAvatar #DigitalHumans #InverseRendering #PhysicsSimulation #AvatarModeling #ClothSimulation #PhotorealisticRendering #ComputerVision #DeepLearning #Innovation


**Some updates on GLiNER**

🆕 A new commercially permissible multilingual version is available urchade/gliner_multiv2.1

🐛 A subtle bug that causes performance degradation on some models has been corrected. Thanks to @yyDing1 for raising the issue.

```
from gliner import GLiNER

# Initialize GLiNER
model = GLiNER.from_pretrained("urchade/gliner_multiv2.1")

text = "This is a text about Bill Gates and Microsoft."

# Labels for entity prediction
labels = ["person", "organization", "email"]

entities = model.predict_entities(text, labels, threshold=0.5)

for entity in entities:
    print(entity["text"], "=>", entity["label"])
```

tomaarsen osanseviero aari1995 Bailefan erickdp mrdbourke


SwapAnything: Enabling Arbitrary Object Swapping in Personalized Visual
  Editing (2404.05717)

SwapAnything is a new method that allows swapping any object in an image with personalized concepts given by a reference image.

Key points:
1⃣ It uses pre-trained diffusion models to enable precise and high-fidelity object swapping in images.
2⃣Targeted variable swapping ensures perfect background preservation while swapping specific areas.
3⃣SwapAnything achieves good results in single-object, multi-object, partial-object, and cross-domain swapping tasks.

Paper: https://huggingface.co/papers/2404.05717
Project page: https://swap-anything.github.io

Congrats to the authors for their work!


Dense Training, Sparse Inference: Rethinking Training of
  Mixture-of-Experts Language Models (2404.05567)

Thrilled to unveil DS-MoE: a dense training and sparse inference scheme for enhanced computational and memory efficiency in your MoE models! 🚀🚀🚀

Discover more in our blog: https://huggingface.co/blog/bpan/ds-moe and dive into the details with our paper: https://huggingface.co/papers/2404.05567

samusenps umuthopeyildirim zoratrox osanseviero victor ybelkada mayank-mishra


More open research updates 🧵

Performance estimation is currently the best way to quantify the impact of data drift on model performance. 💡

I've been benchmarking performance estimation methods (CBPE and M-CBPE) against data drift signals.

I'm using drift results as features for many regression algorithms, and then I'm taking those to estimate the model's performance. Finally, I'm measuring the Mean Absolute Error (MAE) between the regression models' predictions and actual performance.

So far, for all my experiments, performance estimation methods do better than drift signals. 👨🔬

Bear in mind that these are some early results, I'm running the flow on more datasets as we speak.

Hopefully, by next week, I will have more results to share 👀


Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs (2404.05719)

Ferret-UI

Grounded Mobile UI Understanding with Multimodal LLMs

https://huggingface.co/papers/2404.05719

Recent advancements in multimodal large language models (MLLMs) have been noteworthy, yet, these general-domain MLLMs often fall short in their ability to comprehend and interact effectively with user interface (UI) screens. In this paper, we present Ferret-UI, a new MLLM tailored for enhanced understanding of mobile UI screens, equipped with referring, grounding, and reasoning capabilities. Given that UI screens typically exhibit a more elongated aspect ratio and contain smaller objects of interest (e.g., icons, texts) than natural images, we incorporate "any resolution" on top of Ferret to magnify details and leverage enhanced visual features. Specifically, each screen is divided into 2 sub-images based on the original aspect ratio (i.e., horizontal division for portrait screens and vertical division for landscape screens). Both sub-images are encoded separately before being sent to LLMs. We meticulously gather training samples from an extensive range of elementary UI tasks, such as icon recognition, find text, and widget listing. These samples are formatted for instruction-following with region annotations to facilitate precise referring and grounding. To augment the model's reasoning ability, we further compile a dataset for advanced tasks, including detailed description, perception/interaction conversations, and function inference. After training on the curated datasets, Ferret-UI exhibits outstanding comprehension of UI screens and the capability to execute open-ended instructions. For model evaluation, we establish a comprehensive benchmark encompassing all the aforementioned tasks. Ferret-UI excels not only beyond most open-source UI MLLMs, but also surpasses GPT-4V on all the elementary UI tasks.

samusenps keenyou fscheffczyk KvrParaskevi


Very excited to share the first two official Gemma variants from Google! Today at Google Cloud Next, we announced cutting-edge models for code and research!

First, https://huggingface.co/collections/google/codegemma-release-66152ac7b683e2667abdee11 -  a new set of code-focused Gemma models at 2B and 7B, in both pretrained and instruction-tuned variants. These exhibit outstanding performance on academic benchmarks and (in my experience) real-life usage. Read more in the excellent HuggingFace blog: https://huggingface.co/blog/codegemma

Second, (https://huggingface.co/collections/google/recurrentgemma-release-66152cbdd2d6619cb1665b7a), which is based on the outstanding Google DeepMind research in Griffin: https://arxiv.org/abs/2402.19427. RecurrentGemma is a research variant that enables higher throughput and vastly improved memory usage. We are excited about new architectures, especially in the lightweight Gemma sizes, where innovations like RecurrentGemma can scale modern AI to many more use cases.

For details on the launches of these models, check out our launch blog -- and please do not hesitate to send us feedback. We are excited to see what you build with CodeGemma and RecurrentGemma!

Huge thanks to the Hugging Face team for helping ensure that these models work flawlessly in the Hugging Face ecosystem at launch!

andysalerno kramp rmdhirr manvinder osanseviero reach-vb philschmid eliebak victor nsarrazin BrigitteTousi m-ric tomaarsen lewtun VictorSanh andrewrreed lysandre Wauplin julien-c mcpotato yjernite lunarflu linoyts Tonic pcuenq Sintayew4 icaio

osanseviero reach-vb philschmid eliebak victor nsarrazin BrigitteTousi m-ric tomaarsen lewtun VictorSanh andrewrreed lysandre Wauplin brunatrevelin julien-c yjernite lunarflu Tonic pcuenq Sintayew4 Blane187 clefourrier

osanseviero reach-vb philschmid victor nsarrazin BrigitteTousi tomaarsen lewtun andrewrreed lysandre Wauplin julien-c lunarflu Tonic pcuenq Blane187 icaio


🎉  Super @DeepMount00  just released 𝗚𝗲𝗺𝗺𝗮_𝗤𝗔_𝗜𝗧𝗔_𝘃𝟯 𝗹𝗲𝗮𝗱𝗶𝗻𝗴 the 𝗥𝗔𝗚 𝘁𝗮𝘀𝗸 on the Italian 𝗟𝗟𝗠_𝗜𝗧𝗔_𝗟𝗘𝗔𝗗𝗘𝗥𝗕𝗢𝗔𝗥𝗗. The model is a fine tuned version of Gemma 2B.
Model details: https://huggingface.co/DeepMount00/Gemma_QA_ITA_v3
Explore the full RAG section rankings here: https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard on section Classifica RAG


🔁 AutoMerger created the best 7B model on the Open LLM Leaderboard

By randomly combining top models from the Open LLM Leaderboard, AutoMerger created YamshadowExperiment28-7B. The model is three weeks old and has been at the top of the leaderboard for a week now. It was created through a simple SLERP merge of:

- https://huggingface.co/automerger/YamShadow-7B (another top model created by AutoMerger)
- https://huggingface.co/yam-peleg/Experiment28-7B (a top model from @yam-peleg)

1/ On the Open LLM Leaderboard, it managed to outperform the excellent M7-7b model, which has been the #1 7B model for a while now.

2/ On the YALL leaderboard, YamshadowExperiment28-7B is ranked as the 9th best-performing automerge (but note that the scores are very close to each other). Compared to others, it does not perform particularly well on AGIEval or Bigbench.

3/ Thanks to @sam-paech , I have scores on EQ-Bench, where it managed to outperform all of my previous models. It even surpasses recent models such as DBRX instruct, Qwen1.5 32B Chat, and Cohere's Command R+.

Surprisingly, it does not support ChatML or Mistral Instruct, unlike my other merges (which are part of its family tree). Alpaca works well 99% of the time, but the model can sometimes produce a lot of "INST" tokens for no reason.

In my experiments, YamshadowExperiment28-7B doesn't seem smarter than other successful merges like AlphaMonarch. On the contrary, I found several mathematical or reasoning problems where it fails.

Considering these results, it looks like it might overfit the Open LLM Leaderboard. I guess it's anything but surprising when you randomly merge 156 models.

🤗 Model: https://huggingface.co/automerger/YamshadowExperiment28-7B
🔁 AutoMerger: https://huggingface.co/spaces/mlabonne/AutoMerger

victor AtAndDev Tonic vladbogo rmdhirr samusenps Gargaz osanseviero kgourgou Kukedlc Ramikan-BR

neeravkumar Heng666 Gargaz osanseviero mylesgoose kgourgou eliolio Muhammad2003 Ramikan-BR


Do language models plan ahead for future tokens? (2404.00859)

🔍 Today's pick in Interpretability & Analysis of LMs: Do language models plan ahead for future tokens? by W. Wu @jxm @lionellevine

This work aims to evaluate whether language models exhibit implicit planning during generation.

Authors propose two hypotheses that could result in planning-like behavior:

- Pre-caching: the model engages in computation that is functional to future, but not current, predictions.

- Breadcrumbs: Features contributing to the current prediction happen to also be the ones improving future ones.

To validate which behavior is observed in practice, authors note that off-diagonal gradients for weight matrices across the model are the ones responsible for pre-caching, and craft a variant of gradient descent (myopic descent) to remove such terms from the optimization procedure.

Using a synthetic dataset, authors demonstrate that pre-caching occurs in Transformers language models. However, for natural language settings the LM is observed to leverage breadcrumbs from previous passes even in the case of myopic training, rendering the latter hypothesis more plausible to account for model behavior.

📄 Paper: https://huggingface.co/papers/2404.00859

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-ofc-lms-65ae3339949c5675d25de2f9


LegalKit Retrieval, a binary Search with Scalar (int8) Rescoring through French legal codes is now available as a 🤗 Space.

This process is designed to be memory efficient and fast, with the binary index being small enough to fit in memory and the int8 index being loaded as a view. Additionally, the binary index is much faster (up to 32x) to search than the float32 index, while the rescoring is also extremely efficient.

This space also showcases the tsdae-lemone-mbert-base, a sentence embedding model based on BERT fitted using Transformer-based Sequential Denoising Auto-Encoder for unsupervised sentence embedding learning with one objective : french legal domain adaptation.

Link to the 🤗 Space : https://huggingface.co/spaces/louisbrulenaudet/legalkit-retrieval

Notes:
The SentenceTransformer model currently in use is in beta and may not be suitable for direct use in production.

colin-r-carter marksmithh osanseviero tomaarsen samusenps ethanker


InstructHumans: Editing Animated 3D Human Textures with Instructions (2404.04037)

🚀💃🌟 New Research Alert (Avatars Collection)! 🌟🕺🚀
📄 Title: InstructHumans: Editing Animated 3D Human Textures with Instructions

📝 Description: InstructHumans is a novel framework for text-instructed editing of 3D human textures that employs a modified Score Distillation Sampling (SDS-E) method along with spatial smoothness regularization and gradient-based viewpoint sampling to achieve high-quality, consistent, and instruction-true edits.

👥 Authors: Jiayin Zhu, Linlin Yang, Angela Yao

🔗 Paper: https://huggingface.co/papers/2404.04037

🌐 Web Page: https://jyzhu.top/instruct-humans
📁 Repository: https://github.com/viridityzhu/InstructHumans

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #InstructHumans #3DTextureEditing #TextInstructions #ScoreDistillationSampling #SDS-E #SpatialSmoothnessRegularization  #3DEditing #AvatarEditing #DeepLearning #Innovation


We just released gradio version 4.26.0 ! We *highly* recommend you upgrade your apps to this version to bring in these nice changes:

🎥 Introducing the API recorder. Any gradio app running 4.26.0 and above will have an "API Recorder" that will record your interactions with the app and auto-generate the corresponding python or js code needed to recreate those actions programmatically. It's very neat!

📝 Enhanced markdown rendering in gr.Chatbot

🐢 Fix for slow load times on spaces as well as the UI locking up on rapid generations

See the full changelog of goodies here: https://www.gradio.app/changelog#4-26-0

samusenps DmitryRyumin dillfrescott gsarti osanseviero aceeee ysharma on1onmangoes


Fun fact about evaluation, part 2!

How much do scores change depending on prompt format choice?

Using different prompts (all present in the literature, from `Prompt question?` to `Question: prompt question?\nChoices: enumeration of all choices\nAnswer: `), we get a score range of...

10 points for a single model!
Keep in mind that we only changed the prompt, not the evaluation subsets, etc.
Again, this confirms that evaluation results reported without their details are basically bullshit.

Prompt format on the x axis, all these evals look at the logprob of either "choice A/choice B..." or "A/B...".

Incidentally, it also changes model rankings - so a "best" model might only be best on one type of prompt...

eduagarcia SivilTaram andysalerno samusenps osanseviero EricNarro


text-generation-inference (TGI) is now fully open-source again!

Along with text-embeddings-inference.

We just switched both of those repos' license back to Apache 2. 🔥

clefourrier Saugatkafley samusenps jcudit flozi00 osanseviero joennlae Joseph717171 clem freddyaboulton

sugatoray gsarti kramp osanseviero Joseph717171 clem timestop1

Sylvestre clefourrier ehcalabres osanseviero Joseph717171 clem


No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency
  Determines Multimodal Model Performance (2404.04125)

No "Zero-Shot" Without Exponential Data

Pretraining Concept Frequency Determines Multimodal Model Performance

https://huggingface.co/papers/2404.04125

Web-crawled pretraining datasets underlie the impressive "zero-shot" evaluation performance of multimodal models, such as CLIP for classification/retrieval and Stable-Diffusion for image generation. However, it is unclear how meaningful the notion of "zero-shot" generalization is for such multimodal models, as it is not known to what extent their pretraining datasets encompass the downstream concepts targeted for during "zero-shot" evaluation. In this work, we ask: How is the performance of multimodal models on downstream concepts influenced by the frequency of these concepts in their pretraining datasets? We comprehensively investigate this question across 34 models and five standard pretraining datasets (CC-3M, CC-12M, YFCC-15M, LAION-400M, LAION-Aesthetics), generating over 300GB of data artifacts. We consistently find that, far from exhibiting "zero-shot" generalization, multimodal models require exponentially more data to achieve linear improvements in downstream "zero-shot" performance, following a sample inefficient log-linear scaling trend. This trend persists even when controlling for sample-level similarity between pretraining and downstream datasets, and testing on purely synthetic data distributions. Furthermore, upon benchmarking models on long-tailed data sampled based on our analysis, we demonstrate that multimodal models across the board perform poorly. We contribute this long-tail test set as the "Let it Wag!" benchmark to further research in this direction. Taken together, our study reveals an exponential need for training data which implies that the key to "zero-shot" generalization capabilities under large-scale training paradigms remains to be found.

samusenps vicgalle vladbogo adhirajghosh KvrParaskevi


✨ Easy Synthetic Dataset File Generation using LLM DataGen ! Link: https://huggingface.co/spaces/lhoestq/LLM_DataGen

features + how it works:

✍ Generate the dataset content you want just by entering a file name
💡 Optionally specify the column names you need
💨 The dataset is streamed and generated on-the-fly in JSON Lines format
✅ Generation is constrained to always output valid JSON

How does this work ?
1/ Enter a file name
2/ The model generates column names for such a file. Using structured generation, it can generate 2 to 5 column names using lower case characters and underscores. I use a prompt that asks to generate column names for a realistic dataset and low temperature.
3/ The columns are used to update the Finite State Machine for the dataset content structured generation, so that it is used to generate JSON objects using those columns
4/ The model generates JSON objects using structured generation again, using the updated Finite State Machine. I use a prompt that asks for realistic data and a temperature of 1.

> Why update a Finite State Machine instead of re-creating one ?

Creating one can take up to 30sec, while updating one takes 0.1s (though it requires to manipulate a graph which is not easy to implement)

> Batched generation is faster, why not use it ?

Generate in batches is faster but tends to generate duplicates for this demo.
Further work can be to provide different prompts (one per sequence in the batch) to end up with a different distribution of sequences in each batch. Or implement a custom sampler that would forbid generating the same data in sequences of the same batch.

> How does structured generation work ?

I used the `outlines` library with `transformers` to to define a JSON schema that the generation has to follow. It uses a Finite State Machine with `token_id` as transitions.

Let me know what you think ! And feel free to duplicate/modify it to try other models/prompts or sampling methods :)


Brain2Music: Reconstructing Music from Human Brain Activity (2307.11078)

Decoding speech from non-invasive brain recordings (2208.12266)

Seeing through the Brain: Image Reconstruction of Visual Perception from
  Human Brain Signals (2308.02510)

DreamDiffusion: Generating High-Quality Images from Brain EEG Signals (2306.16934)

NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual
  Pretraining and Multi-level Modulation (2403.18211)

NeuroCine: Decoding Vivid Video Sequences from Human Brain Activties (2402.01590)

Cinematic Mindscapes: High-quality Video Reconstruction from Brain
  Activity (2305.11675)

MinD-3D: Reconstruct High-quality 3D objects in Human Brain (2312.07485)

DreamGaussian4D: Generative 4D Gaussian Splatting (2312.17142)

Sora: A Review on Background, Technology, Limitations, and Opportunities
  of Large Vision Models (2402.17177)

Hello world!

I'd like to share with you all today some specific Research about the Brain & some surface level thoughts

Music (Frequencies)
https://huggingface.co/papers/2307.11078

Speech
https://huggingface.co/papers/2208.12266

Image
https://huggingface.co/papers/2308.02510
https://huggingface.co/papers/2306.16934
https://huggingface.co/papers/2403.18211

Video
https://huggingface.co/papers/2402.01590
https://huggingface.co/papers/2305.11675

3D
https://huggingface.co/papers/2312.07485

Potential Opportunities For BCI

4D
https://huggingface.co/papers/2312.17142

Realistic High Quality Video
https://huggingface.co/papers/2402.17177

https://huggingface.co/collections/samusenps/bci-661206b642da659656474db2

Reading minds is cool & useful but could be utilized for many things other than thought interrogation 👀

Current Co pilots are Boring! & have much untapped potential & also people don't seem to want autonomous agents replacing them although it is inevitable for some cases

I believe if humans want to become an interplanetary species that can utilize our accumulative research we need to extend our brain with technology in order to be smarter. Imagine a Co-pilot for your head, Adding extra ‘RAM’ to the brain, or even Processing external data within the brain.

Ok people are afraid of implanting computer chips within their brains, what if someone hacks it ? , the invasive possibilities are crazy!

How can we ensure Safety & Interpretability in brain computer interfaces
1. External Non Invasive Brain Computer interfaces [ think similar to https://neurosity.co/crown (overpriced IMO & Hardware is closed source proprietary, who knows what they’re doing 👁) ]
2. Full Reproducible Open-Source Stack Brain computer interface down to the hardware, operating system, and application level.
3. Maybe you can't, there may always be a risk of danger, though not as consequential with 1 & 2

samusenps INMYEYES DmitryRyumin Jigya gsarti diwank osanseviero lunarflu victor medmac01 not-lain s3nh efecelik

digiplay osanseviero flyineye lunarflu medmac01 not-lain s3nh


3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (2312.09228)

🚀🕺🌟 New Research Alert - CVPR 2024 (Avatars Collection)! 🌟💃🚀
📄 Title: 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting 🔝

📝 Description: 3DGS-Avatar is a novel method for creating animatable human avatars from monocular videos using 3D Gaussian Splatting (3DGS). By using a non-rigid deformation network and as-isometric-as-possible regularizations, the method achieves comparable or better performance than SOTA methods while being 400x faster in training and 250x faster in inference, allowing real-time rendering at 50+ FPS.

👥 Authors: Zhiyin Qian, Shaofei Wang, Marko Mihajlovic, Andreas Geiger, Siyu Tang

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2312.09228

🌐 Github Page: https://neuralbodies.github.io/3DGS-Avatar/
📁 Repository: https://github.com/mikeqzy/3dgs-avatar-release

📺 Video: https://www.youtube.com/watch?v=FJ29U9OkmmU

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #3DGSAvatar #3DAvatar #3DGaussianSplatting #AnimatableAvatars #MonocularVideo #RealTimeRendering #FastTraining #EfficientInference #CVPR2024 #DeepLearning #Innovation

DmitryRyumin AlekseiPravdin Jigya YoungMetal yehiaserag dillfrescott osanseviero Lei-bw Redcalx34B mariasandu lognat0704

samusenps AlekseiPravdin Jigya YoungMetal dillfrescott osanseviero Lei-bw Redcalx34B lognat0704

DmitryRyumin Jigya YoungMetal dillfrescott


ReFT: Representation Finetuning for Language Models (2404.03592)

🔍 Today's pick in Interpretability & Analysis of LMs: ReFT: Representation Finetuning for Language Models by @zhengxuanzenwu @aryaman Z. Wang @atticusg D. Jurafsky @manning @cgpotts

This work introduces Representation fine-tuning (ReFT), a framework using learned inference-time interventions as efficient yet effective alternatives to PEFT weight adaptation. LoReFT, a ReFT variant intervening linearly on a representation subspaces, is evaluated against several PEFT approaches showing SOTA performances across popular benchmark with 10-50x speedup. The 🤗-compatible pyreft library is introduced to simplify ReFT usage.

This is one of the most convincing practical applications of interpretability methods/insights I've seen in recent years, and I'm looking forward to people combining this with methods to disentangle features like SAEs and Backpack LMs for making interventions more interpretable!

📄 Paper: https://huggingface.co/papers/2404.03592

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9

samusenps aryaman Jigya zhengxuanzenwu diwank srisree


To date, louisbrulenaudet/Maxine-34B-stock is the "Best 🤝 base merges and moerges model of around 30B" on the Open LLM Leaderboard ❤🔥

It is a practical application of the stock method recently implemented by @arcee-ai in the MergeKit :
```yaml
models:
    - model: ConvexAI/Luminex-34B-v0.2
    - model: fblgit/UNA-34BeagleSimpleMath-32K-v1
merge_method: model_stock
base_model: abacusai/Smaug-34B-v0.1
dtype: bfloat16
```
Model : https://huggingface.co/louisbrulenaudet/Maxine-34B-stock
LLM Leaderboard best models ❤🔥 Collection : https://huggingface.co/collections/open-llm-leaderboard/llm-leaderboard-best-models-652d6c7965a4619fb5c27a03

samusenps dhjang ben-wise Jigya osanseviero not-lain


Releasing H2O-Danube2-1.8b - a new and improved language model being the current top model on the Open LLM Leaderboard.

The model is trained on additional 2T tokens (total of 3T tokens) with various data mix stages as a result of extensive experimentation. Focus is on natural language tasks.

Base weights: https://huggingface.co/h2oai/h2o-danube2-1.8b-base
SFT weights: https://huggingface.co/h2oai/h2o-danube2-1.8b-sft
Chat weights: https://huggingface.co/h2oai/h2o-danube2-1.8b-chat


Honestly i don't understand how come we as the open source community haven't surpassed GPT-4 yet ? Like for me it looks like everything is out there just need to be exploited! Clearly specialized small models outperforms gpt4 on downstream tasks ! So why haven't we just trained a 1B-2B really strong general model and then continue pertained and/or finetuned it on datasets for downstream tasks like math, code...well structured as Textbooks format or other datasets formats that have been proven to be really efficient and good! Ounce you have 100 finetuned model, just wrap them all into a FrankenMoE and Voila ✨
And that's just what a NOOB like myself had in mind, I'm sure there is better, more efficient ways to do it ! So the question again, why we haven't yet ? I feel I'm missing something... Right?

samusenps awhsnap victor m18coppola visitthething


Very interesting model just released by MyShell: https://huggingface.co/jetmoe/jetmoe-8b . It's a 8B-parameters MoE LLM so 2.2B active parameters, really efficient.

Main characteristics:
- impressive performances for its size (beating https://huggingface.co/meta-llama/Llama-2-7b and https://huggingface.co/huggyllama/llama-13b)
- combine Mixture of Attention heads (MoA) and Mixture of MLP Experts (MoE) – 8 experts with 2 being active for each token
- trained on a rather limited 1.25T tokens from publicly available datasets – training recipe follows the MiniCPM's two-phases training method => first time I see this for a 2B+ model
- $100k to train
- open weights - open sharing of recipes - open dataset - open code => ♡
- still interesting room to improve performances (be it only by training longer)

Links:
- report: https://research.myshell.ai/jetmoe
- model: https://huggingface.co/jetmoe/jetmoe-8b
- code: https://github.com/myshell-ai/JetMoE

Note: I actually detailed all of the MiniCPM schedule, Mixture-of-expert (MoE) and many of the datasets used in this work in my recent little guide to building LLMs in 2024, so feel free to check it out if you want to learn more on these topics: https://www.youtube.com/watch?v=2-SPH9hIKT8

samusenps a9i giux78 dillfrescott clem osanseviero

samusenps ajibawa-2023 dillfrescott clem osanseviero


🐦 The IBIS Challenge: an open competition in Inferring and predicting transcription factor Binding Specificities: modeling DNA patterns recognized by human regulatory proteins.

🧬 Deciphering human gene regulation is a cornerstone of modern molecular biology and biomedicine. Gene activity is controlled by special regulatory proteins, the transcription factors, which recognize DNA sequence patterns. We invite you to join IBIS in our search for the best method to model binding specificities of yet unexplored human regulatory proteins.

In the challenge, you may use classic methods to represent sequence patterns or any modern approaches
🚀 including decision trees, CNNs, RNNs, LSTMs, and transformers.

💡IBIS allows using arbitrary genome or random sequences to pre-train an artificial neural network or to extract features, and use a few existing datasets, see the IBIS documentation (https://ibis.autosome.org/docs/technical_details).

📊 Yet, the main power and opportunity come with a diverse array of experimental data on 40 human regulatory proteins, many of which remained unexplored until now.

🏆 The best methods will be highlighted in the post-challenge high-impact scientific paper 📝, while the winners 🥇of the Primary track of the Final round will be invited to contribute as co-authors.

🌐 Learn more at https://ibis.autosome.org/
🤗 Our article at HF: https://huggingface.co/blog/nikgr/the-ibis-challenge
👥 Organizers - GRECO-BIT & Codebook consortia: https://ibis.autosome.org/docs/about_us


Sparse Feature Circuits: Discovering and Editing Interpretable Causal
  Graphs in Language Models (2403.19647)

🔍 Today's pick in Interpretability & Analysis of LMs: Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models by @sammarks C. Rager @eircjm @belinkov @davidbau @amueller

This work proposes using features and errors from sparse autoencoders trained to reconstruct LM activations as interpretable units for circuit discovery. The authors then introduce SHIFT, a technique for editing model behavior by ablating interpretable elements from sparse feature circuits. This method is applied alongside unsupervised circuit discovery at scale by means of clustering, showing highly interpretable feature circuits interacting to produce behaviors like predicting sequence increments.

I found the experiment of Section 4 especially convincing and exciting in terms of downstream applications: authors trained a classifier over a biased dataset, and showcased how SHIFT intervention in feature space leads to performances matching those of the same model trained on an unbiased data distribution!

📄 Paper: https://huggingface.co/papers/2403.19647

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9

samusenps nikgr thomwolf osanseviero victor toyxyz


Today we're starting a new initiative: LM Studio Community Models! 🤖

@bartowski, a prolific quantizer (both GGUF and EXL2) will be helping to curate notable new models in LM Studio's Community Models page: https://huggingface.co/lmstudio-community.

Our goal is to ensure the community has access to GGUF files for new & noteworthy models as soon as possible. Keep an eye on that page for updates.

If you're unfamiliar with GGUF, it's the de-facto standard for 'compressed' LLM weights. It is the native format of llama.cpp (https://github.com/ggerganov/llama.cpp, an LLM runtime C/C++ library.) This format is supported in LM Studio.

We will also be sharing new models on the LM Studio Discord: https://discord.gg/aPQfnNkxGC

bartowski victor samusenps Kukedlc InferenceIllusionist bunnycore Severian osanseviero MexIvanov dillfrescott nikgr thomwolf lixin67 julien-c sroecker Midgardsormr AgentHaswing apol British-Rat tcowley


TIL: since Text Generation Inference supports Messages API, which is compatible with the OpenAI Chat Completion API, you can trace calls made to inference endpoints using Langfuse's OpenAI API integration.

A Hugging Face Pro subscription includes access to many models you want to test when developing an app (https://huggingface.co/blog/inference-pro). Using the endpoint and tracing your generations during this development process is an excellent way for GPU-poor people to bootstrap an initial dataset quickly while prototyping.

samusenps lunarflu osanseviero nikgr victor VictorSanh


After giving GPU Programming a hands-on try, I have come to appreciate the level of complexity in AI compute:

- Existing/leading frameworks (CUDA, OpenCL, DSLs, even Triton), still fall at the mercy of low-level compute that requires deeper understanding and experience.
- Ambiguous optimizations methods that will literally drive you mad 🤯
- Triton is cool but not cool enough (high level abstractions that fall back to low level compute issues as you build more specialized kernels)
- As for CUDA, optimization requires considering all major components of the GPU (DRAM, SRAM, ALUs) 🤕
- Models today require stallion written GPU kernels to reduce storage and compute cost.
- GPTQ was a big save 👍🏼

@karpathy  is right expertise in this area is scarce and the reason is quite obvious - uncertainties: we are still struggling to get peak performance from multi-connected GPUs while maintaining precision and reducing cost.

May the Scaling Laws favor us lol.

samusenps lunarflu badayvedat Abecid axwinters


📄 𝗣𝗮𝗽𝗲𝗿 𝗵𝗲𝗿𝗲 👉

📄 𝗣𝗮𝗽𝗲𝗿 𝗵𝗲𝗿𝗲 👉

Mixture-of-Depths: Dynamically allocating compute in transformer-based
  language models (2404.02258)

[𝐍𝐞𝐰 𝐏𝐚𝐩𝐞𝐫] 𝐀𝐥𝐥 𝐭𝐨𝐤𝐞𝐧𝐬 𝐬𝐡𝐨𝐮𝐥𝐝 𝐧𝐨𝐭 𝐫𝐞𝐪𝐮𝐢𝐫𝐞 𝐭𝐡𝐞 𝐬𝐚𝐦𝐞 𝐞𝐟𝐟𝐨𝐫𝐭 𝐭𝐨 𝐜𝐨𝐦𝐩𝐮𝐭𝐞! ⇒ 𝐌𝐢𝐱𝐭𝐮𝐫𝐞 𝐨𝐟 𝐝𝐞𝐩𝐭𝐡𝐬 🫧🐠

Google Researchers were unhappy with the way current decoding generally works: all tokens go through the same layers, thus requiring exactly the same effort to compute.

Whereas in reality, completing the answer to a difficult math problem for instance should be more computationally intense than completing the text of the Declaration of Independence: 𝗻𝗼𝘁 𝗮𝗹𝗹 𝘁𝗼𝗸𝗲𝗻𝘀 𝗮𝗿𝗲 𝗰𝗿𝗲𝗮𝘁𝗲𝗱 𝗲𝗾𝘂𝗮𝗹!

➡ 𝗧𝗵𝗲𝘆 𝗵𝗮𝗱 𝘁𝗵𝗶𝘀 𝗴𝗲𝗻𝗶𝘂𝘀 𝗶𝗱𝗲𝗮: 💡 𝗵𝗮𝘃𝗶𝗻𝗴 𝗮 𝘁𝗼𝗸𝗲𝗻 𝗴𝗼 𝘁𝗵𝗿𝗼𝘂𝗴𝗵 𝗮 𝗯𝗹𝗼𝗰𝗸 𝘀𝗵𝗼𝘂𝗹𝗱 𝗯𝗲 𝗼𝗽𝘁𝗶𝗼𝗻𝗮𝗹. The token can go through the block (thus undergoing expensive self-attention computation) or avoid it through a skip connection.
The routing decision is taken on the block level: each block selects from the total sequence the top-k tokens that will go through it, and the others tokens will skip it. 𝘛𝘩𝘪𝘴 𝘢𝘭𝘭𝘰𝘸𝘴 𝘵𝘰 𝘤𝘩𝘰𝘰𝘴𝘦 𝘵𝘩𝘦 𝘦𝘹𝘢𝘤𝘵 𝙘𝙖𝙥𝙖𝙘𝙞𝙩𝙮 𝘰𝘧 𝘢 𝘣𝘭𝘰𝘤𝘬, 𝘪.𝘦. 𝘵𝘩𝘦 𝘱𝘳𝘰𝘱𝘰𝘳𝘵𝘪𝘰𝘯 𝘰𝘧 𝘵𝘰𝘬𝘦𝘯𝘴 𝘵𝘩𝘢𝘵 𝘨𝘰 𝘵𝘩𝘳𝘰𝘶𝘨𝘩 𝘪𝘵, 𝘸𝘩𝘪𝘤𝘩 𝘥𝘪𝘳𝘦𝘤𝘵𝘭𝘺 𝘪𝘯𝘧𝘭𝘶𝘦𝘯𝘤𝘦𝘴 𝘵𝘩𝘦 𝘤𝘰𝘮𝘱𝘶𝘵𝘢𝘵𝘪𝘰𝘯𝘢𝘭 𝘪𝘯𝘵𝘦𝘯𝘴𝘪𝘵𝘺 𝘰𝘧 𝘵𝘩𝘦 𝘧𝘰𝘳𝘸𝘢𝘳𝘥 𝘱𝘢𝘴𝘴.

This yields Mixture-of-Depths (MoD), with spectacular results.

✨ 𝗥𝗲𝘀𝘂𝗹𝘁𝘀:
🎚 𝗖𝗮𝗽𝗮𝗰𝗶𝘁𝘆 𝗰𝗮𝗻 𝗯𝗲 𝘁𝘂𝗻𝗲𝗱 𝗮𝗹𝗹 𝘁𝗵𝗲 𝘄𝗮𝘆 𝗱𝗼𝘄𝗻 𝘁𝗼 𝟭𝟮.𝟱% for every second block: thus 87.5% of tokens just skip the block!
🚀 For the same training time and performance, >𝟲𝟬% 𝗶𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝘀𝗽𝗲𝗲𝗱!
🤝 𝗖𝗮𝗻 𝗯𝗲 𝗰𝗼𝗺𝗯𝗶𝗻𝗲𝗱 𝘄𝗶𝘁𝗵 𝗠𝗶𝘅𝘁𝘂𝗿𝗲-𝗼𝗳-𝗘𝘅𝗽𝗲𝗿𝘁𝘀 for further improvements.

📄 𝗣𝗮𝗽𝗲𝗿 𝗵𝗲𝗿𝗲 👉 https://huggingface.co/papers/2404.02258
📚 I added it to my paper collection 👉 https://huggingface.co/collections/m-ric/spinning-up-in-llms-659e698f9dd5a71bd3f579a7

clem samusenps ABX-AI mmhamdy nikgr minjejeon qinluo MultiTrickFox mammour danielus Joseph717171 ivanfioravanti


Sailor: Open Language Models for South-East Asia (2404.03608)

⚓ Sailor: A New Multilingual Open LLM for South-East Asia 🌏

Last month we have released a new family of multilingual language models called **Sailor**, ranging from 0.5B to 7B parameters, continually pre-trained from the Qwen1.5 models. Based on our extensive benchmarking, the Sailor models demonstrate exceptional performance on South-East Asian languages, taking us one step closer to multilingual LLMs that can serve the diverse needs of the region and beyond.

Today, we're more than excited to share the key technical details behind the Sailor models! 💪

**Key highlights**:
🔍 Data curation: Merging short examples, document-level code-switching, aggressive data cleaning and deduplication.
🤖 Tokenization Robustness: We find that BPE dropout is really effective to deal with prompt variations.
🔍 Optimizing Data Mixture: We propose a new approach to automatically balance capabilities across different languages!
🌟 Recipe in Continual Pre-training: We discover a powerful metric that can help predict how well the Sailor models will perform on the original domain (e.g., English) after continual pre-training.

We are thrilled to share these technical details with the community and invite you to explore the Sailor models. We hope Sailor models take us one step closer to multilingual LLMs in the world! 🌍✨

To learn more, please access our research paper or reach out to our team.
🔗 Paper: https://huggingface.co/papers/2404.03608
🧩 Model: https://huggingface.co/collections/sail/sailor-language-models-65e19a749f978976f1959825
💻 Code: https://github.com/sail-sg/sailor-llm

dreamerdeo Jiiiia teddythinh chaoscodes DmitryRyumin taufiqdp SivilTaram osanseviero clem victor lunarflu Kernel nikgr gabrielchua DongfuJiang damerajee yjernite

dreamerdeo Jiiiia teddythinh aboutyourresearch chaoscodes monsoon-nlp osanseviero clem SivilTaram samusenps lunarflu damerajee yjernite


Mixture-of-Depths

Dynamically allocating compute in transformer-based language models

https://huggingface.co/papers/2404.02258

Transformer-based language models spread FLOPs uniformly across input sequences. In this work we demonstrate that transformers can instead learn to dynamically allocate FLOPs (or compute) to specific positions in a sequence, optimising the allocation along the sequence for different layers across the model depth. Our method enforces a total compute budget by capping the number of tokens (k) that can participate in the self-attention and MLP computations at a given layer. The tokens to be processed are determined by the network using a top-k routing mechanism. Since k is defined a priori, this simple procedure uses a static computation graph with known tensor sizes, unlike other conditional computation techniques. Nevertheless, since the identities of the k tokens are fluid, this method can expend FLOPs non-uniformly across the time and model depth dimensions. Thus, compute expenditure is entirely predictable in sum total, but dynamic and context-sensitive at the token-level. Not only do models trained in this way learn to dynamically allocate compute, they do so efficiently. These models match baseline performance for equivalent FLOPS and wall-clock times to train, but require a fraction of the FLOPs per forward pass, and can be upwards of 50\% faster to step during post-training sampling.

samusenps taufiqdp clem mick-bitovi iancos csouza nikgr rupeshs sugatoray dillfrescott ceshine


Introducing https://huggingface.co/datasets/gretelai/synthetic_text_to_sql by https://huggingface.co/gretelai

It stands as the largest and most diverse synthetic Text-to-SQL dataset available to-date.

The dataset includes:

- 105,851 records partitioned into 100,000 train and 5,851 test records
~23M total tokens, including ~12M SQL tokens
- Coverage across 100 distinct domains/verticals
- Comprehensive array of SQL tasks: data definition, retrieval, manipulation, analytics & reporting
- Wide range of SQL complexity levels, including subqueries, single joins, multiple joins, aggregations, window functions, set operations
- Database context, including table and view create statements
- Natural language explanations of what the SQL query is doing
- Contextual tags to optimize model training

Blogpost: https://gretel.ai/blog/synthetic-text-to-sql-dataset
Dataset: https://huggingface.co/datasets/gretelai/synthetic_text_to_sql

ichigoberry mvaloatto Svngoku nikgr thomwolf lhoestq brhnsbn


GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from
  a Single Image (2404.02152)

🚀🎭🌟 New Research Alert - CVPR 2024 (Avatars Collection)! 🌟 🎭🚀
📄 Title: GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image 🔝

📝 Description: GeneAvatar is a generic approach for editing 3D head avatars based on a single 2D image, applicable to different volumetric representations. The novel expression-aware generative modification model delivers high quality and consistent editing results across multiple viewpoints and emotions.

👥 Authors: Chong Bao et al.

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2404.02152

🌐 Github Page: https://zju3dv.github.io/geneavatar/
📁 Repository: https://github.com/zju3dv/GeneAvatar

📺 Video: https://www.youtube.com/watch?v=4zfbfPivtVU

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #GeneAvatar #HeadAvatar #3DHeadAvatarEditing #VolumetricHeadAvatar #SingleImageEditing #ExpressionAwareModification #CVPR2024 #DeepLearning #Innovation

dashfunnydashdash samusenps Limbicnation MichaelKwak osanseviero clem introvoyz041 Benson Alkohole

DmitryRyumin samusenps osanseviero Kukedlc clem


Introducing the Gradio API Recorder 🪄

Every Gradio app now includes an API recorder that lets you reconstruct your interaction in a Gradio app as code using the Python or JS clients! Our goal is to make Gradio the easiest way to build ML APIs, not just UIs 🔥

Bagus victor merve gabrielchua samusenps radames WhatIFTheMind osanseviero ademait brianjking RocktProductions

DmitryRyumin merve nikgr samusenps radames jeffboudier osanseviero

DmitryRyumin victor merve radames osanseviero


InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image
  Generation (2404.02733)

Greeting!

We are happy to introduce our InstantStyle, which is a framework that employs straightforward yet potent techniques for achieving effective disentanglement of style and content from reference images.

Code: https://github.com/InstantStyle/InstantStyle
Paper: https://huggingface.co/papers/2404.02733
Project Page: https://instantstyle.github.io/

DmitryRyumin samusenps victor merve tonyassi namelysweet thomwolf badayvedat dillfrescott penouc


ARABIC CHINESE FRENCH GERMAN RUSSIAN SPANISH TURKISH

mLLM - first release:
orca_dpo_pairs by Intel (translated into 7 languages)

ARABIC CHINESE FRENCH GERMAN RUSSIAN SPANISH TURKISH

Upcoming:
- more datasets
- cleaning steps
- a blogpost
- stay updated at https://hf.co/multilingual

https://huggingface.co/datasets/multilingual/orca_dpo_pairs

samusenps monsoon-nlp seyf1elislam aari1995 osanseviero victor merve nikgr Kukedlc


🆕 Releasing a new series of 8 zeroshot classifiers: better performance, fully commercially useable thanks to synthetic data, up to 8192 tokens, run on any hardware.

Summary:
🤖 The zeroshot-v2.0-c series replaces commercially restrictive training data with synthetic data generated with mistralai/Mixtral-8x7B-Instruct-v0.1 (Apache 2.0). All models are released under the MIT license.
🦾 The best model performs 17%-points better across 28 tasks vs. facebook/bart-large-mnli (the most downloaded commercially-friendly baseline).
🌏 The series includes a multilingual variant fine-tuned from BAAI/bge-m3 for zeroshot classification in 100+ languages and with a context window of 8192 tokens
🪶 The models are 0.2 - 0.6 B parameters small, so they run on any hardware. The base-size models are +2x faster than bart-large-mnli while performing significantly better.
🤏 The models are not generative LLMs, they are efficient encoder-only models specialized in zeroshot classification through the universal NLI task.
🤑 For users where commercially restrictive training data is not an issue, I've also trained variants with even more human data for improved performance.

Next steps:
✍ I'll publish a blog post with more details soon
🔮 There are several improvements I'm planning for v2.1. Especially the multilingual model has room for improvement.

All models are available for download in this Hugging Face collection: https://huggingface.co/collections/MoritzLaurer/zeroshot-classifiers-6548b4ff407bb19ff5c3ad6f

These models are an extension of the approach explained in this paper, but with additional synthetic data: https://arxiv.org/abs/2312.17543

victor tomaarsen samusenps clem vladbogo jeffboudier mmhamdy osanseviero peter2000 merve peycke


👋 👋 👋  In a recent AI Time Journal Interview, I was asked:  How has HackerNoon leveraged AI to enhance its platform? and what lessons have you learned about AI’s role in spam creation and prevention?

https://HackerNoon.com  uses AI and machine learning to improve grammar, translate stories, generate featured images, suggest possible headlines, (and note, I would love to hear what else you think we should use AI for!) . Maybe Clippy was ahead of its time? Writers have and will continue to have smart AI assistants. AI can not and should not replace the writer, but it should and can assist the writer and editor throughout the publishing process. We are documenting our approach to editorial at https://EditingProtocol.com. With the rise of LLMs, we’ve seen an influx of AI story spam submissions. Other blogging platforms are hosting so many of these types of machine-generated stories right now. We are actively taking steps not to become a wasteland of AI’s musings. Because every story is reviewed before publication (with about half being rejected), I think our average story quality is higher than other blogging platforms. Every story submission automatically enters into plagiarism and AI writing detection. We highlight what sections are likely, possibly, and unlikely to be written by AI. It’s all about confidence levels. Ultimately, human editors make the final call in interpreting the machines’ reports.


Advancing LLM Reasoning Generalists with Preference Trees (2404.02078)

Advancing LLM Reasoning Generalists with Preference Trees

https://huggingface.co/papers/2404.02078

We introduce Eurus, a suite of large language models (LLMs) optimized for reasoning. Finetuned from Mistral-7B and CodeLlama-70B, Eurus models achieve state-of-the-art results among open-source models on a diverse set of benchmarks covering mathematics, code generation, and logical reasoning problems. Notably, Eurus-70B beats GPT-3.5 Turbo in reasoning through a comprehensive benchmarking across 12 tests covering five tasks, and achieves a 33.3% pass@1 accuracy on LeetCode and 32.6% on TheoremQA, two challenging benchmarks, substantially outperforming existing open-source models by margins more than 13.3%. The strong performance of Eurus can be primarily attributed to UltraInteract, our newly-curated large-scale, high-quality alignment dataset specifically designed for complex reasoning tasks. UltraInteract can be used in both supervised fine-tuning and preference learning. For each instruction, it includes a preference tree consisting of (1) reasoning chains with diverse planning strategies in a unified format, (2) multi-turn interaction trajectories with the environment and the critique, and (3) pairwise data to facilitate preference learning. UltraInteract allows us to conduct an in-depth exploration of preference learning for reasoning tasks. Our investigation reveals that some well-established preference learning algorithms may be less suitable for reasoning tasks compared to their effectiveness in general conversations. Inspired by this, we derive a novel reward modeling objective which, together with UltraInteract, leads to a strong reward model.

samusenps clem vladbogo merve nikgr Bruno


Introducing VLRM: Vision-Language Models act as Reward Models for Image Captioning
Paper:
https://arxiv.org/abs/2404.01911
Model weights and training logs:
https://huggingface.co/sashakunitsyn/vlrm-blip2-opt-2.7b

We propose VLRM, a method for fine-tuning an existing image captioning model using reinforcement learning and vision-language models as reward models.
The method manages to significantly improve the generation quality without human-labeled data and is applicable to any image captioning model.
Our model reaches impressive 0.90 R@1 CLIP Recall score on MS-COCO Carpathy Test Split.

samusenps nikgr victor clem osanseviero merve Kukedlc AlkaouDembele xziayro


Efficient 3D Implicit Head Avatar with Mesh-anchored Hash Table
  Blendshapes (2404.01543)

🚀🎭🌟 New Research Alert - CVPR 2024 (Avatars Collection)! 🌟 🎭🚀
📄 Title: MonoAvatar++: Efficient 3D Implicit Head Avatar with Mesh-anchored Hash Table Blendshapes 🔝

📝 Description: MonoAvatar++ is a real-time neural implicit 3D head avatar model with high quality and fine-grained control over facial expressions. It uses local hash table blendshapes attached to a parametric facial model for efficient rendering, achieving SOTA results even for challenging expressions.

👥 Authors: Ziqian Bai, Feitong Tan, Sean Fanello, Rohit Pandey, Mingsong Dou, Shichen Liu, Ping Tan, Yinda Zhang

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2404.01543

🌐 Github Page: https://augmentedperception.github.io/monoavatar-plus

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #MonoAvatar++ #HeadAvatar #3DModeling #AvatarGeneration #NeuralImplicitAvatar #EfficientRendering #CVPR2024 #DeepLearning #Innovation

DmitryRyumin samusenps nikgr victor clem merve


ChemLLM version 1.5 has been open sourced. It has also released a version that has undergone human feedback reinforcement learning. In particular, the front-end and back-end codes have been open sourced simultaneously to facilitate deployment. Everyone is welcome to pay attention.
The evaluation set and part of the training set will also be contributed to the community in days, so stay tuned ✧٩(ˊωˋ*)و✧

ChemLLM-1.5
https://huggingface.co/AI4Chem/ChemLLM-7B-Chat-1.5-DPO

ChemLLM-1.5-dpo https://huggingface.co/AI4Chem/ChemLLM-7B-Chat-1.5-SFT

Front-end https://github.com/AI4Chem/NextChat
RAG application example https://github.com/AI4Chem/NextChat/blob/main/chatpdf.py
Backend https://github.com/AI4Chem/api_backend

Front-end Demo https://chemllm.org/
RAG application example Demo https://chatpdf.chemllm.org/
Backend Demo https://api.chemllm.org/

qq8933 osanseviero ajibawa-2023 samusenps nikgr clem merve mmhamdy dillfrescott


Following up on @vikhyatk's Moondream2 update and @santiagomed's implementation on Candle, I quickly put togheter the WASM module so that you could try running the ~1.5GB quantized model in the browser. Perhaps the next step is to rewrite it using https://github.com/huggingface/ratchet and run it even faster with WebGPU, @FL33TW00D-HF.

https://huggingface.co/spaces/radames/Candle-Moondream-2

ps: I have a collection of all Candle WASM demos here https://huggingface.co/collections/radames/candle-wasm-examples-650898dee13ff96230ce3e1f

vikhyatk FL33TW00D-HF samusenps nikgr clem merve Felladrin osanseviero

vikhyatk FL33TW00D-HF clem merve kylen osanseviero


Gradio 4.25.0 is out with some nice improvements and bug fixes!

🧹 Automatic deletion of gr.State variables stored in the server. Never run out of RAM again. Also adds an unload event you can run when a user closes their browser tab.

😴 Lazy example caching. You can set cache_examples="lazy" to cache examples when they're first requested as opposed to before the server launches. This can cut down the server's start-up time drastically.

🔊 Fixes a bug with streaming audio outputs

🤖 Improvements to gr.ChatInterface like pasting images directly from the clipboard.

See the rest of the changelog here: https://www.gradio.app/changelog#4-25-0

samusenps ysharma not-lain ajibawa-2023 radames DmitryRyumin nikgr clem clefourrier akhaliq on1onmangoes


Released a new version of https://huggingface.co/vikhyatk/moondream2 today! Primarily focused on improving OCR and captioning (e.g. "Describe this image", "Describe this image in one sentence"), but also seeing general improvement across all benchmarks.

not-lain Corianas ajibawa-2023 radames DmitryRyumin nikgr clem xingren23 dillfrescott aaryanverma linz Jorgegarciairazabal Vlansu

not-lain clem sooperDime Sylvestre dillfrescott omaryshchenko Jorgegarciairazabal


These 15 open models are available for serverless inference on Cloudflare Workers AI, powered by GPUs distributed in 150 datacenters globally - 👏 @rita3ko @mchenco @jtkipp @nkothariCF @philschmid

https://huggingface.co/collections/Cloudflare/hf-curated-models-available-on-workers-ai-66036e7ad5064318b3e45db6

rwightman nkothariCF samusenps ajibawa-2023 philschmid radames victor nikgr kramp clem Tonic

samusenps philschmid radames victor clem Tonic


Anthropic introduces "Many-shot Jailbreaking" (MSJ), a new attack on large language models! MSJ exploits long context windows to override safety constraints.

Key Points:
* Prompts LLMs with hundreds of examples of harmful behavior formatted as a dialogue
* Generates malicious examples using an uninhibited "helpful-only" model
* Effective at jailbreaking models like Claude 2.0, GPT-3.5, GPT-4
* Standard alignment techniques provide limited protection against long context attacks

Paper: https://www.anthropic.com/research/many-shot-jailbreaking
More details in my blog: https://huggingface.co/blog/vladbogo/many-shot-jailbreaking

Congrats to the authors for their work!


𝟐𝟎𝟐𝟒, 𝐭𝐡𝐞 𝐲𝐞𝐚𝐫 𝐨𝐟 𝐚𝐠𝐞𝐧𝐭 𝐰𝐨𝐫𝐤𝐟𝐥𝐨𝐰𝐬 🔧🦾🤖

𝟐𝟎𝟐𝟒, 𝐭𝐡𝐞 𝐲𝐞𝐚𝐫 𝐨𝐟 𝐚𝐠𝐞𝐧𝐭 𝐰𝐨𝐫𝐤𝐟𝐥𝐨𝐰𝐬 🔧🦾🤖

𝗪𝗵𝘆 𝘂𝘀𝗲 𝗮𝗴𝗲𝗻𝘁 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀?

𝗪𝗵𝘆 𝘂𝘀𝗲 𝗮𝗴𝗲𝗻𝘁 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀?

𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗱𝗲𝘀𝗶𝗴𝗻 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀

𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗱𝗲𝘀𝗶𝗴𝗻 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀

𝗖𝗹𝗼𝘀𝗶𝗻𝗴 𝘁𝗵𝗼𝘂𝗴𝗵𝘁𝘀

𝗖𝗹𝗼𝘀𝗶𝗻𝗴 𝘁𝗵𝗼𝘂𝗴𝗵𝘁𝘀

𝟐𝟎𝟐𝟒, 𝐭𝐡𝐞 𝐲𝐞𝐚𝐫 𝐨𝐟 𝐚𝐠𝐞𝐧𝐭 𝐰𝐨𝐫𝐤𝐟𝐥𝐨𝐰𝐬 🔧🦾🤖

I've just watched Andrew Ng's talk at Sequoia last week.
If you're interested in Agents, you should really watch it!

𝗪𝗵𝘆 𝘂𝘀𝗲 𝗮𝗴𝗲𝗻𝘁 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀?
The current LLM task solving workflow is not very intuitive:
We ask it “write an essay all in one shot, without ever using backspace.”

Why not allow the LLM a more similar process to what we would do?
- “Write an essay outline”
- “Do you need wen research?”
- “Write a first draft”
- “Consider improvements”
…

This is called an Agentic workflow. Existing ones bring a huge performance boost. With HumanEval: GPT-4 zero-shot gets 67% score, agentic with either one of tool use or reflection goes over 90%, and the combination of the two scores even higher!

𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗱𝗲𝘀𝗶𝗴𝗻 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀
On the following two points, the tech is robust:

⚙ 𝗥𝗲𝗳𝗹𝗲𝘅𝗶𝗼𝗻: For instance: add a critic step after the writing step
🛠 𝗧𝗼𝗼𝗹 𝘂𝘀𝗲: extends the capabilities of the LLM by allowing it to call tools, like search or calculator

The next two will be needed to go further, but the tech for them is more emerging and not reliable yet:
🗺 𝗣𝗹𝗮𝗻𝗻𝗶𝗻𝗴 forward to decompose task into subtasks. This allows great behaviours like an AI Agent re-routing after a failure
🐝 𝗠𝘂𝗹𝘁𝗶-𝗮𝗴𝗲𝗻𝘁 𝗰𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝗼𝗻: Program a flock of agents with tasks.
Improving the two above points will unlock huge performance boosts!

Andrew NG says Research agents are already part of his workflow!

𝗖𝗹𝗼𝘀𝗶𝗻𝗴 𝘁𝗵𝗼𝘂𝗴𝗵𝘁𝘀
Andrew speculates that through agentic workflows, maybe generating many tokens fast from a small LLM will give better results than slower throughput from a powerful LLM like GPT-5.

🎬 Watch the talk here 👉 https://www.youtube.com/watch?v=sal78ACtGTc
📚 I've added his recommended reads to https://huggingface.co/collections/m-ric/agents-65ba776fbd9e29f771c07d4e

osanseviero clem neelam91721 samusenps nikgr JoPmt Moibe abidlabs poGlingus


Aurora-M: The First Open Source Multilingual Language Model Red-teamed
  according to the U.S. Executive Order (2404.00399)

Aurora-M

The First Open Source Multilingual Language Model Red-teamed according to the U.S. Executive Order

https://huggingface.co/papers/2404.00399

Pretrained language models underpin several AI applications, but their high computational cost for training limits accessibility. Initiatives such as BLOOM and StarCoder aim to democratize access to pretrained models for collaborative community development. However, such existing models face challenges: limited multilingual capabilities, continual pretraining causing catastrophic forgetting, whereas pretraining from scratch is computationally expensive, and compliance with AI safety and development laws. This paper presents Aurora-M, a 15B parameter multilingual open-source model trained on English, Finnish, Hindi, Japanese, Vietnamese, and code. Continually pretrained from StarCoderPlus on 435 billion additional tokens, Aurora-M surpasses 2 trillion tokens in total training token count. It is the first open-source multilingual model fine-tuned on human-reviewed safety instructions, thus aligning its development not only with conventional red-teaming considerations, but also with the specific concerns articulated in the Biden-Harris Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. Aurora-M is rigorously evaluated across various tasks and languages, demonstrating robustness against catastrophic forgetting and outperforming alternatives in multilingual settings, particularly in safety evaluations.

osanseviero CulturedMan DmitryRyumin samusenps clem Catliba mayank-mishra sugatoray


How would you benchmark performance estimation algorithms vs data drift signals?

I'm working on a benchmarking analysis, and I'm currently doing the following:

-  Get univariate and multivariate drift signals and measure their correlation with realized performance.
- Use drift signals as features of a regression model to predict the model's performance.
- Use drift signals as features of a classification model to predict a performance drop.
- Compare all the above experiments with results from Performance Estimation algorithms.

Any other ideas?


🥇Open CoT Leaderboard

We're delighted to announce the [Open CoT Leaderboard](https://huggingface.co/spaces/logikon/open_cot_leaderboard) on 🤗 Spaces.

Unlike other LLM performance leaderboards, the Open CoT Leaderboard is not tracking absolute benchmark accuracies, but relative **accuracy gains** due to **chain-of-thought**.

Eval datasets that underpin the leaderboard are hosted [here](https://huggingface.co/cot-leaderboard).

Feedback and suggestions more than welcome.

@clefourrier

clefourrier lunarflu osanseviero samusenps giux78 davidhought0n reuank victor nikgr yakazimir clem


We would like to announce our Aurora-M multilingual models which is based on Starcoderplus.
Twitter: https://twitter.com/ontocord/status/1772778544051155029
LinkedIn: https://www.linkedin.com/feed/update/urn:li:activity:7178521998845759488/
Blog post: https://huggingface.co/blog/mayank-mishra/aurora
Arxiv: https://huggingface.co/papers/2404.00399

Current LLMs are very susceptible to generating toxic, harmful and even dangerous content. They can also generate outputs with gender or racial biases. The Biden-Harris Executive Order https://www.federalregister.gov/documents/2023/11/01/2023-24283/safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence) sets forth guidelines on what is considered a safe AI system.
Following up on these guidelines, we present the world's first open source Biden-Harris Executive Order Red teamed Multilingual Language Model: Aurora-M. Inspired by BigScience, the model is trained on 5 languages: English, Hindi, Japanese, Vietnamese and Finnish.

* Red teamed model: https://huggingface.co/aurora-m/aurora-m-biden-harris-redteamed(safety tuned according to the order mentioned above)
* Base model: https://huggingface.co/aurora-m/aurora-m-base (not safety tuned)
* Instruct model: https://huggingface.co/aurora-m/aurora-m-instruct (not safety tuned)

@mayank-mishra @cabbage972 @sted97 @Xa9aX @Taishi-N324 @Muennighoff @vumichien @prateeky2806 @felfri @spyysalo and many many others!

samusenps Xa9aX lunarflu osanseviero mayank-mishra vumichien kenhktsui nikgr sted97 muhtasham Taishi-N324 Gokumase

samusenps Xa9aX lunarflu osanseviero mayank-mishra vumichien kenhktsui sted97 Taishi-N324


⚡ AutoQuant

AutoQuant is the evolution of my previous AutoGGUF notebook (https://colab.research.google.com/drive/1P646NEg33BZy4BfLDNpTz0V0lwIU3CHu). It allows you to quantize your models in five different formats:

- GGUF: perfect for inference on CPUs (and LM Studio)
- GPTQ/EXL2: fast inference on GPUs
- AWQ: super fast inference on GPUs with vLLM (https://github.com/vllm-project/vllm)
- HQQ: extreme quantization with decent 2-bit and 3-bit models

Once the model is converted, it automatically uploads it on the Hugging Face Hub. To quantize a 7B model, GGUF only needs a T4 GPU, while the other methods require an A100 GPU.

Here's an example of a model I quantized using HQQ and AutoQuant: https://huggingface.co/mlabonne/AlphaMonarch-7B-2bit-HQQ

I hope you'll enjoy it and quantize lots of models! :)

💻 AutoQuant: https://colab.research.google.com/drive/1b6nqC7UZVt8bx4MksX7s656GXPM-eWw4

marcsun13 mishig Outrun32 johko kgourgou DataSoul samusenps lunarflu osanseviero julien-c OxxoCodes musfiqdehan seyf1elislam nasiruddin15 Heng666 nikgr sbarman25 matlok reach-vb AlekseiPravdin maaarcooo aspenita victor sbrandeis Kukedlc

eldogbbhed kgourgou louisbrulenaudet reach-vb multitude0099 DavidAU AlekseiPravdin ManniX-ITA nguyenthanhdo Kukedlc


Symbol-LLM: Towards Foundational Symbol-centric Interface For Large
  Language Models (2311.09278)

Check out our work Symbol-LLM ! We have open-sourced both 7B / 13B model weights, as well as part of the symbolic collections ! Try it !

Paper link: https://huggingface.co/papers/2311.09278

Model weights: https://huggingface.co/Symbol-LLM/Symbol-LLM-7B-Instruct

Symbol-LLM xufangzhi FeYuan vladbogo lunarflu osanseviero nikgr

xufangzhi FeYuan QiushiSun lunarflu osanseviero


Introducing Indic Chat!

Try out best opensource Indic LLMs now on https://www.indic.chat/

Models available:
• Telugu-LLM-Labs/Indic-gemma-7b-finetuned-sft-Navarasa-2.0
• GenVRadmin/AryaBhatta-GemmaOrca
• BhabhaAI/Gajendra-v0.1
• ai4bharat/Airavata

Additionally:

1. We open up our discord for everyone to collaborate & accelerate Indic LLMs: https://bhabha.ai/discord

2. We release ~600K rows filtered & Hindi translated version of OpenHermes-2.5 instruction dataset: https://huggingface.co/datasets/BhabhaAI/openhermes-2.5-hindi

Also, thanks to our compute sponsor - Telugu LLM Labs & Bhabha AI in helping us serve models for Indic Chat.

If you’d like to be a sponsor too, checkout
https://www.indic.chat/sponsor

monsoon-nlp lunarflu osanseviero alielfilali01 ajibawa-2023 nikgr ashimksaha Saugatkafley

samusenps lunarflu osanseviero alielfilali01


On evaluating fine tuned 7B Italian open source LLMs I have collected many data points and I created a super simple explorative analyses. My hypothesis  based on data are:

 - mmlu is hard to improve when fine tuning a base model on a different language
- fine tuning also on single GPUs can improve by 5% to 10% the base model on common tasks but a lot more on specific cases with the right training time and data
- fine tuning can specialize well but at cost of loosing some foundational  knowledge.

Here the data https://docs.google.com/spreadsheets/d/1MBcxy1loK8eIycZG4DN84Q2ejZ0jSjxUBgoShHDR6IY/edit?usp=sharing
Here the colab https://colab.research.google.com/drive/1ra4_skG5QYWSYOzvagOoIoj4bibQD8Gw?usp=sharing
Here an article with some considerations https://medium.com/@giuxale/an-analyses-on-italian-llms-models-evaluations-51bffe1d44d1


Gecko: Versatile Text Embeddings Distilled from Large Language Models (2403.20327)

Google DeepMind introduces Gecko a new text embedding! Gecko uses a two-step process that leverages synthetic data generation and reranking.

Keypoints:
* Uses an LLM to generate diverse synthetic queries and tasks from web passages
* Refines the data by retrieving candidate passages and relabeling positives/negatives using the same LLM
* Achieves very good results on the Massive Text Embedding Benchmark, where compact 256D Gecko outperforms 768D models.
* 768D Gecko achieves state-of-the-art performance competing with models a lot larger larger.

Paper: https://huggingface.co/papers/2403.20327
More details in my blog:  https://huggingface.co/blog/vladbogo/gecko

Congrats to the authors for their work!

samusenps clem monsoon-nlp beekay24 mohammedbriman lunarflu nikgr tatv047 super-cinnamon


Jamba: A Hybrid Transformer-Mamba Language Model (2403.19887)

Jamba

A Hybrid Transformer-Mamba Language Model

https://huggingface.co/papers/2403.19887

We present Jamba, a new base large language model based on a novel hybrid Transformer-Mamba mixture-of-experts (MoE) architecture. Specifically, Jamba interleaves blocks of Transformer and Mamba layers, enjoying the benefits of both model families. MoE is added in some of these layers to increase model capacity while keeping active parameter usage manageable. This flexible architecture allows resource- and objective-specific configurations. In the particular configuration we have implemented, we end up with a powerful model that fits in a single 80GB GPU. Built at large scale, Jamba provides high throughput and small memory footprint compared to vanilla Transformers, and at the same time state-of-the-art performance on standard language model benchmarks and long-context evaluations. Remarkably, the model presents strong results for up to 256K tokens context length. We study various architectural decisions, such as how to combine Transformer and Mamba layers, and how to mix experts, and show that some of them are crucial in large scale modeling. We also describe several interesting properties of these architectures which the training and evaluation of Jamba have revealed, and plan to release checkpoints from various ablation runs, to encourage further exploration of this novel architecture. We make the weights of our implementation of Jamba publicly available under a permissive license.

samusenps osanseviero Benyucong clem beekay24 mohammedbriman vladbogo adhi01 lunarflu nikgr kramp ABX-AI