🎯🖼🌟 New Research Alert - ICLR 2024! 🌟 🖼🎯
📄 Title: Adversarial AutoMixup 🖼

📝 Description: Adversarial AutoMixup is an approach to image classification augmentation. By alternately optimizing a classifier and a mixed-sample generator, it attempts to generate challenging samples and improve the robustness of the classifier against overfitting.

👥 Authors: Huafeng Qin et al.

📅 Conference: ICLR, May 7-11, 2024 | Vienna, Austria 🇦🇹

🔗 Paper: https://huggingface.co/papers/2312.11954

📁 Repository: https://github.com/JinXins/Adversarial-AutoMixup

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🔍 Keywords: #AutoMixup #ImageClassification #ImageAugmentation #AdversarialLearning #ICLR2024 #DeepLearning #Innovation

samusenps osanseviero clem beekay24 lunarflu atrxial MiSTe-R nikgr PantheonAI


Little know gem: the Open-source Cookbook

A collection of notebooks for building practical AI applications using open-source tools and models: https://lnkd.in/e6m6Jmwu

Doc: https://lnkd.in/e3FE6TUq

Currently contains 16 notebooks in English (and some in Chinese):
	1.	Using LLM-as-a-judge 🧑⚖ for an automated and versatile evaluation
	2.	Create a legal preference dataset
	3.	Suggestions for Data Annotation with SetFit in Zero-shot Text Classification
	4.	Implementing semantic cache to improve a RAG system
	5.	Building A RAG Ebook “Librarian” Using LlamaIndex
	6.	Stable Diffusion Interpolation
	7.	Building A RAG System with Gemma, MongoDB and Open Source Models
	8.	Prompt Tuning with PEFT Library
	9.	Migrating from OpenAI to Open LLMs Using TGI’s Messages API
	10.	Automatic Embeddings with TEI through Inference Endpoints
	11.	Simple RAG for GitHub issues using Hugging Face Zephyr and LangChain
	12.	Embedding multimodal data for similarity search using 🤗 transformers, 🤗 datasets and FAISS
	13.	Fine-tuning a Code LLM on Custom Code on a single GPU
	14.	RAG Evaluation Using Synthetic data and LLM-As-A-Judge
	15.	Advanced RAG on HuggingFace documentation using LangChain
	16.	Detecting Issues in a Text Dataset with Cleanlab

giux78 samusenps osanseviero clem beekay24 ajibawa-2023 vladbogo peaceAsh lunarflu nikgr vikas

samusenps giux78 osanseviero clem lunarflu anandamide Srjnnnn OmbelineM


The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits (2402.17764)

SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation (2309.17448)

Diaries of Open Source. Part 13 🤗

🤏Two different bitnet 1.5 open-source replications
Original paper: https://hf.co/papers/2402.17764
1bitllm experiment: https://hf.co/blog/joey00072/experiments-with-bitnet-1-5
NousResearch experiment https://hf.co/NousResearch/OLMo-Bitnet-1B

 🥳Tiny and large multimodal models great for embeddings
GitHub: https://github.com/unum-cloud/uform
Encoders: https://hf.co/collections/unum-cloud/multimodal-encoders-660553903617c5297eb16838
ONNX weights: https://hf.co/collections/unum-cloud/uform-vl-english-large-onnx-66055a57c182d846f3bc1949

📜 SMPLer-X: Expressive Human Pose and Shape Estimation
Project website: https://caizhongang.com/projects/SMPLer-X/
Demo: https://huggingface.co/spaces/caizhongang/SMPLer-X
Paper: https://hf.co/papers/2309.17448

🧙GeoWizard: 3D Geometry Estimation
Project website: https://fuxiao0719.github.io/projects/geowizard/
Demo: https://hf.co/spaces/lemonaddie/geowizard

Misc models and datasets
- Dolphin-2.8-mistral-7b-v0.2 https://hf.co/cognitivecomputations/dolphin-2.8-mistral-7b-v02
- Hermes-2-Pro-11B,  a self-frankenmerge 11B variant https://hf.co/mattshumer/Hermes-2-Pro-11B
- Large conversational dataset based on Usenet data in the Italian language https://hf.co/datasets/mii-community/UsenetArchiveIT-conversations

ruggsea mrinaldi DmitryRyumin samusenps giux78 clem vladbogo lunarflu nikgr abidlabs


This is the closest I’ve seen of a scalable AI/LLM Operating System - it has all the major ingredients of a feasible AI OS 1 architecture:

- Extends classical OS functionalities with an LLM Kernel.
- Multi agent-centric approach.
- Optimized resource allocation system that allows for LLM-based tasks and Classical OS tasks to coexist.
- An Agent Scheduler that can perform classical os operations (FIFO, RR).
- A Context Manager to improve alignment.
- Lazy Memory Manager for agents (ensures data is stored and accessible only while the agent is active)
- An Enhanced security module for the AI-driven environment.

It does hit all checkpoints, doesn’t it? An upscale version of @karpathy’s.

Code: https://github.com/agiresearch/AIOS

ajibawa-2023 osanseviero phuong-d-h-nguyen Dlbk rsamantha KvrParaskevi AIAdventurist JoshuaGu NeuralNovel thomwolf kazeMace disham993 kubo-vn abhinit21 clem jhondavid DataSoul alielfilali01 lunarflu nikgr Skier8402

samusenps tatoy JoseRFJunior thomwolf Nakini disham993 lunarflu

chuangxinlezhi disham993 KIKININA lunarflu


🚀🚀 Exciting times for the document AI community!

We're thrilled to announce the release of some of the largest OCR datasets available to the public.
🔥 With over 26 million pages , 18 billion text tokens, and 6TB of data, these resources are a significant leap forward for document AI research.

Here's how to access these datasets quickly:

```
from datasets import load_dataset

pdfa_dataset = load_dataset('pixparse/pdfa-eng-wds', streaming=True)
IDL_dataset = load_dataset('pixparse/idl-wds', streaming=True)
```
This enables you to stream them directly, integrating seamlessly with your projects using the Hugging Face datasets library. On the hub, you can find them here:

https://huggingface.co/datasets/pixparse/pdfa-eng-wds
https://huggingface.co/datasets/pixparse/idl-wds

For lean data loading, the new [chug](https://github.com/huggingface/chug) library offers a solution with pdf decoding:

```
import chug

task_cfg = chug.DataTaskDocReadCfg(
    page_sampling='all',
)
data_cfg = chug.DataCfg(
    source='pixparse/pdfa-eng-wds',
    split='train',
    batch_size=None,
    format='hfids',
    num_workers=0,
)
data_loader = chug.create_loader(
    data_cfg,
    task_cfg,
)
sample = next(iter(data_loader))
```

We owe a huge thank you to Peter Wyatt, Kate Tasker, Rachel Taketa, Ali Furkan Biten, Ruben Tito, and their colleagues for their contributions. Their work putting these datasets together has been invaluable. 🤗

Looking Ahead:

We're on a mission to enhance document AI capabilities, and these datasets are just the beginning. With your engagement and innovation, we're confident in the community's ability to develop robust OCR solutions. We encourage you to explore these datasets, experiment with the code, and contribute to the collective progress in document AI.

For detailed information on usage and licensing, please refer to the dataset cards on the Hugging Face hub.

abidlabs VictorSanh lhoestq Dlbk thomwolf manishiitg clefourrier IlyasMoutawwakil osanseviero samusenps DmitryRyumin ajibawa-2023 Pclanglais loubnabnl rwightman marioparreno Chunte NeuralNovel AIAdventurist JoshuaGu antfraia mmhamdy callmeraxz qinluo pankaj-munde Zyn123 andrewrreed ABX-AI pcuenq giux78 victor clem tonnirobo mohammedbriman Wauplin julien-c lunarflu Tom-Neverwinter nikgr kramp francisrafal MoritzLaurer aceeee

rwightman abidlabs VictorSanh lhoestq Dlbk thomwolf IlyasMoutawwakil osanseviero loubnabnl Chunte mmhamdy andrewrreed ABX-AI pcuenq clem Wauplin julien-c lunarflu Tom-Neverwinter

rwightman abidlabs VictorSanh lhoestq Dlbk thomwolf IlyasMoutawwakil osanseviero loubnabnl Chunte andrewrreed ABX-AI clem Jakaline Wauplin julien-c lunarflu

lhoestq Dlbk thomwolf IlyasMoutawwakil osanseviero Chunte femboysLover andrewrreed ABX-AI Wauplin julien-c lunarflu prithivMLmods

shiv2050 Chunte ssml2050 visisco andrewrreed ABX-AI Wauplin julien-c lunarflu Tom-Neverwinter

samusenps Chunte andrewrreed ABX-AI Wauplin julien-c lunarflu Tom-Neverwinter Tonic


Diaries of Open Source. Part 12 🤗

🚀Alibaba releases Qwen1.5-MoE-A2.7B, an interesting MoE with 2.7B activated parameters and 64 experts
Blog https://qwenlm.github.io/blog/qwen-moe/
Demo: https://hf.co/spaces/Qwen/qwen1.5-MoE-A2.7B-Chat-demo
Models: https://hf.co/Qwen
GitHub: https://github.com/QwenLM/Qwen1.5

🎵VoiceCraft, SOTA speech editing and text to speech
GitHub: https://github.com/jasonppy/VoiceCraft
Model: https://huggingface.co/pyp1/VoiceCraft

🐍 AI21Labs release Jamba, an SSM-Transformer, pretrained MoE which allows a large context window (256K) and high throughput
Blog https://www.ai21.com/blog/announcing-jamba
Model https://huggingface.co/ai21labs/Jamba-v0.1

✨ Berkeley releases Starling-LM-7B, an RLHF-ed model, and -RM-34B, a Yi-based reward model very good for its size
Starling Beta: https://hf.co/Nexusflow/Starling-LM-7B-beta
Starling RM: https://hf.co/Nexusflow/Starling-RM-34B

🖥Stability releases Stable Code Instruct 3B, an instruct model for code generation
Blog: https://stability.ai/news/introducing-stable-code-instruct-3b
Demo: https://hf.co/spaces/stabilityai/stable-code-instruct-3b
Report: https://stability.ai/s/Stable_Code_TechReport_release.pdf

📚Common Corpus: the largest public domain dataset for training LLMs
Blog: https://hf.co/blog/Pclanglais/common-corpus
Dataset: https://hf.co/collections/PleIAs/common-corpus-65d46e3ea3980fdcd66a5613

Misc:
⚡GaLore: a very memory-efficient technique that allows pretraining models in consumer GPUs https://hf.co/blog/galore
Moirai
📈Moirai, foundation models for time series forecasting https://hf.co/collections/Salesforce/moirai-10-r-models-65c8d3a94c51428c300e0742
🔥 Mistral-ORPO-Capybara-7K, a high-quality Mistral fine-tune using ORPO, a new alignment technique https://hf.co/kaist-ai/mistral-orpo-capybara-7k
🤯APISR, an anime super-resolution upscaling model https://hf.co/spaces/HikariDawn/APISR

samusenps Dlbk manishiitg Joseph717171 DmitryRyumin ajibawa-2023 HikariDawn ElenaRyumina Chunte gabrielmbmb mmhamdy callmeraxz clem lunarflu nikgr mlabonne JustinLin610

samusenps Dlbk Joseph717171 HikariDawn Chunte mmhamdy clem Titus-von-Koeller lunarflu JustinLin610


Happy to share Living Images and the demo video of the product outpainting model behind it 🚀

Send your generation requests in the thread 🧵or use https://img.coframe.ai

AlekseyKorshuk Glavin001 tunahfish onum TamasZumpf clem lunarflu nikgr qyxs


Fun fact about evaluation!

Did you know that, if you evaluate the same model, with the same prompt formatting & the same fixed few-shot examples, only changing
♻the order in which the few shot examples are added to the prompt ♻
you get a difference of up to 3 points in evaluation score?

I did a small experiment using some MMLU subsets on the best performing 7B and lower pretrained models from the leaderboard.

I tried 8 different prompting methods (containing more or less information, such as just the question, or Question: question, or Question: question Choices: ..., see the x axis) that are commonly used in evaluation.

I then compared the results for all these methods, in 5-shot, during 2 runs. The *only difference* between the first and second run being that the samples used in few-shot are not introduced in the same order.
For example, run one would have been "A B C D E Current sample", vs, in run 2, "D C E A B Current sample".
All the other experiment parameters stayed exactly the same.

As you can see on the attached picture, you get a difference of up to 3 points between the 2 few-shot samples shuffling.

So, when just changing *the order of the few shot samples* can change your results by several points, what is the impact of all other "minimal" and unreported prompting changes?

-> Any kind of model score, provided without an evaluation script for reproducibility, is basically bullshit (or coms).
-> This is why we need reproducible evaluation in a fair and exactly similar setup, using evaluation suites such as `lm_eval` from the Harness, `lighteval` from HF, or the Open LLM Leaderboard.

shiv2050 ssml2050 InferenceIllusionist mmhamdy ricfergas giux78 clem lunarflu nikgr


Teraflop AI is excited to help support the Caselaw Access Project and Harvard Library Innovation Lab, in the release of over 6.6 million state and federal court decisions published throughout U.S. history. It is important to democratize fair access to data to the public, legal community, and researchers. This is a processed and cleaned version of the original CAP data.

During the digitization of these texts, there were erroneous OCR errors that occurred. We worked to post-process each of the texts for model training to fix encoding, normalization, repetition, redundancy, parsing, and formatting.

Teraflop AI’s data engine allows for the massively parallel processing of web-scale datasets into cleaned text form.

Link to the processed dataset: https://huggingface.co/datasets/TeraflopAI/Caselaw_Access_Project

The Caselaw Access Project dataset is licensed under the CC0 License.

We plan to release trillions of commercially licensed text tokens, images, audio, videos, and other datasets spanning numerous domains and modalities over the next months. If you are interested in contributing commercially licensed data be sure to reach out: https://twitter.com/EnricoShippole

Follow us for the next collaborative dataset releases: https://twitter.com/TeraflopAI

osanseviero samusenps rsamantha Pclanglais muhtasham ssml2050 clem nikgr


Hey, I just added three useful advanced use cases to https://huggingface.co/datasets/do-me/SemanticFinder/blob/main/README.md#advanced-use-cases.
SemanticFinder is a collection of embeddings for public documents or books. You can create your own index file from any text or pdf and save it without installing or downloading anything. Try yourself:

1. Translating from 100+ languages to English (even though it might confuse a strawberry with a grapefruit ;D): https://do-me.github.io/SemanticFinder/?hf=List_of_the_Most_Common_English_Words_70320cde&firstOnly=true&inferencingActive=False
2. Finding English synonyms: https://do-me.github.io/SemanticFinder/?hf=List_of_the_Most_Common_English_Words_0d1e28dc&firstOnly=true&inferencingActive=False
3. The "universal index idea": create an embedding index with 30k English words and reuse it on unseen texts. You can decide to fill the gaps in the index by additional inferencing or just stick to the 30k index for instant semantic similarity.
Initial idea: https://github.com/do-me/SemanticFinder/discussions/48
Try here: https://do-me.github.io/SemanticFinder/?hf=List_of_the_Most_Common_English_Words_0d1e28dc&inferencingActive=False&universalIndexSettingsWordLevel with a text of your choice.

This could be enhanced by adding duplets or triplets like "climate change" or "green house gas". Eventually I'd like to set up vector DB integrations.

Super happy to hear your feedback, ideas and maybe even contributions! :)

---
Edit: Apparently markdown url formatting does only work for HF links.


# PyTorch == 2.5.0 Breaks Transformers' SDPAttention!

When you encounter "RuntimeError: cuDNN Frontend error: [cudnn_frontend] Error: No execution plans support the graph."

We can use workaround like this:

```python
torch.backends.cuda.enable_cudnn_sdp(False)
```

but this slow downs the performance gain from PyTorch 2.5.

Although it is fixed(not "fixed" but default option is turn-off the cuDNN SDPA) at here -- https://github.com/pytorch/pytorch/pull/138587 , but not released yet. (you need to install directly from source)

Fastest way for now : pip install "torch<2.5"

Ref: https://github.com/huggingface/diffusers/issues/9704#issuecomment-2422585273


The Mystery Bot 🕵♂ saga I posted about from earlier this week has been solved...🤗

Cohere for AI has just announced its open source Aya Expanse multilingual model. The Initial release supports 23 languages with more on the way soon.🌌 🌍

You can also try Aya Expanse via SMS on your mobile phone using the global WhatsApp number or one of the initial set of country specific numbers listed below.⬇

🌍WhatsApp - +14313028498
Germany - (+49) 1771786365
USA – +18332746219
United Kingdom — (+44) 7418373332
Canada – (+1) 2044107115
Netherlands – (+31) 97006520757
Brazil — (+55) 11950110169
Portugal – (+351) 923249773
Italy – (+39) 3399950813
Poland - (+48) 459050281

louisbrulenaudet kassiepdj KvrParaskevi TipsUp Felladrin


Good folks at @nvidia have released exciting new research on normalized Transformers (nGPT) for faster and more efficient language modeling!

Here is what they are proposing:

1. Remove all normalization layers, like RMSNorm or LayerNorm, from the standard Transformer architecture.

2. Normalize all matrices along their embedding dimension after each training step. This includes input and output embeddings, attention matrices (Q, K, V), output projection matrices, and MLP matrices.

3. Replace the standard residual connections with normalized update equations using learnable eigen learning rates for the attention and MLP blocks.

4. Change the softmax scaling factor in the attention mechanism from 1/sqrt of d_k to sqrt of d_k.

5. Implement rescaling and optional normalization of query (q) and key (k) vectors in the attention mechanism using learnable scaling factors.

6. Rescale the intermediate states of the MLP block using learnable scaling factors.

7. Implement rescaling of the output logits using learnable scaling factors.

8. Remove weight decay and learning rate warmup from the optimization process.

9. Initialize the eigen learning rates and scaling factors with appropriate values as specified in the paper.

10. During training, treat all vectors and matrices as residing on a unit hypersphere, interpreting matrix-vector multiplications as cosine similarities.

11. Implement the update equations for the hidden states using the normalized outputs from attention and MLP blocks, controlled by the eigen learning rates.

12. After each forward pass, normalize all parameter matrices to ensure they remain on the unit hypersphere.

13. Use the Adam optimizer without weight decay for training the model.

14. When computing loss, apply the learnable scaling factor to the logits before the softmax operation.

15. During inference, follow the same normalization and scaling procedures as in training.

Excited to see how it scales to larger models and datasets!


LoRA with  code 🚀 using PEFT (parameter efficient fine-tuning)

LoRA (Low-Rank Adaptation)
LoRA adds low-rank matrices to specific layers and reduce the number of trainable parameters for efficient fine-tuning.

Code:
Please install these libraries first:
pip install peft
pip install datasets
pip install transformers

```
from transformers import AutoModelForSequenceClassification, Trainer, TrainingArguments
from peft import LoraConfig, get_peft_model
from datasets import load_dataset

# Loading the pre-trained BERT model
model = AutoModelForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2)

# Configuring the LoRA parameters
lora_config = LoraConfig(
    r=8,
    lora_alpha=16,
    lora_dropout=0.1,
    bias="none"
)

# Applying LoRA to the model
model = get_peft_model(model, lora_config)

# Loading dataset for classification
dataset = load_dataset("glue", "sst2")
train_dataset = dataset["train"]

# Setting the training arguments
training_args = TrainingArguments(
    output_dir="./results",
    per_device_train_batch_size=16,
    num_train_epochs=3,
    logging_dir="./logs",
)

# Creating a Trainer instance for fine-tuning
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
)

# Finally we can fine-tune the model
trainer.train()
```

LoRA adds low-rank matrices to fine-tune only a small portion of the model and reduces training overhead by training fewer parameters.
We can perform efficient fine-tuning with minimal impact on accuracy and its suitable for large models where full-precision training is still feasible.


Just released a dataset with 7000+ hours of synthetically generated lo-fi music. https://huggingface.co/datasets/vikhyatk/lofi


I think Reinforcement Learning is the future, for a lot of reasons. I spell them out for you in this video, and also provide you with the basic code to get up and running with Atari and OpenAI Gym. If you want to get into RL, this is your ticket. Link to a cool training montage of the model in the description of the video as well. Step 2 from here would be the full-on training and certification that HuggingFace offers for RL.

https://youtu.be/ueZl3A36ZQk


Finding the Best SmolLM for Your Project

Need an LLM assistant but unsure which hashtag#smolLM to run locally? With so many models available, how can you decide which one suits your needs best? 🤔

If the model you’re interested in is evaluated on the Hugging Face Open LLM Leaderboard, there’s an easy way to compare them: use the model Comparator tool: https://huggingface.co/spaces/open-llm-leaderboard/comparator
Let’s walk through an example👇

Let’s compare two solid options:
- Qwen2.5-1.5B-Instruct from Alibaba Cloud Qwen (1.5B params)
- gemma-2-2b-it from Google (2.5B params)

For an assistant, you want a model that’s great at instruction following. So, how do these two models stack up on the IFEval task?

What about other evaluations?
Both models are close in performance on many other tasks, showing minimal differences. Surprisingly, the 1.5B Qwen model performs just as well as the 2.5B Gemma in many areas, even though it's smaller in size! 📊

This is a great example of how parameter size isn’t everything. With efficient design and training, a smaller model like Qwen2.5-1.5B can match or even surpass larger models in certain tasks.

Looking for other comparisons? Drop your model suggestions below! 👇

John6666 victor pawelg27 Sebuzdugan krishaamer AtAndDev


🌍 I’ve always had a dream of making AI accessible to everyone, regardless of location or language. However, current open MLLMs often respond in English, even to non-English queries!

🚀 Introducing Pangea: A Fully Open Multilingual Multimodal LLM supporting 39 languages! 🌐✨

https://neulab.github.io/Pangea/
https://arxiv.org/pdf/2410.16153

The Pangea family includes three major components:
🔥 Pangea-7B: A state-of-the-art multilingual multimodal LLM capable of 39 languages! Not only does it excel in multilingual scenarios, but it also matches or surpasses English-centric models like Llama 3.2, Molmo, and LlavaOneVision in English performance.

📝 PangeaIns: A 6M multilingual multimodal instruction tuning dataset across 39 languages. 🗂 With 40% English instructions and 60% multilingual instructions, it spans various domains, including 1M culturally-relevant images sourced from LAION-Multi. 🎨

🏆 PangeaBench: A comprehensive evaluation benchmark featuring 14 datasets in 47 languages. Evaluation can be tricky, so we carefully curated existing benchmarks and introduced two new datasets: xChatBench (human-annotated wild queries with fine-grained evaluation criteria) and xMMMU (a meticulously machine-translated version of MMMU).

Check out more details: https://x.com/xiangyue96/status/1848753709787795679

AdinaY John6666 victor apol Felladrin oceansweep KhangHatto nbroad oumayma03


🇫🇷 Lancement officiel de l'OpenLLM French Leaderboard : initiative open-source pour référencer l’évaluation des LLMs francophones

Après beaucoup d’efforts et de sueurs avec Alexandre Lavallee, nous sommes ravis d’annoncer que le OpenLLMFrenchLeaderboard est en ligne sur Hugging Face (space url: https://huggingface.co/spaces/le-leadboard/OpenLLMFrenchLeaderboard) la toute première plateforme dédiée à l’évaluation des grands modèles de langage (LLM) en français. 🇫🇷✨

Ce projet de longue haleine est avant tout une œuvre de passion mais surtout une nécessité absolue. Il devient urgent et vital d'oeuvrer à plus de transparence dans ce domaine stratégique des LLM dits multilingues. La première pièce à l'édifice est donc la mise en place d'une évaluation systématique et systémique des modèles actuels et futurs.

Votre modèle IA français est-il prêt à se démarquer ? Soumettez le dans notre espace, et voyez comment vous vous comparez par rapport aux autres modèles.

❓ Comment ça marche :
Soumettez votre LLM français pour évaluation, et nous le testerons sur des benchmarks de référence spécifiquement adaptés pour la langue française — notre suite de benchmarks comprend :

- BBH-fr : Raisonnement complexe
- IFEval-fr : Suivi d'instructions
- GPQA-fr : Connaissances avancées
- MUSR-fr : Raisonnement narratif
- MATH_LVL5-fr : Capacités mathématiques
- MMMLU-fr : Compréhension multitâche

Le processus est encore manuel, mais nous travaillons sur son automatisation, avec le soutien de la communauté Hugging Face.

@clem  , on se prépare pour une mise à niveau de l’espace ? 😏👀

Ce n'est pas qu'une question de chiffres—il s'agit de créer une IA qui reflète vraiment notre langue, notre culture et nos valeurs. OpenLLMFrenchLeaderboard est notre contribution personnelle pour façonner l'avenir des LLM en France.

clem AdinaY John6666 EnguerrandLec louisbrulenaudet MyDsoElliott coyotte508 malhajar MaziyarPanahi

clem PeteMarc davanstrien malhajar Jeronymous MaziyarPanahi


Crossed more than 100+ of the best and most interesting LoRAs from open-source contributors, based on Flux Dev and Schnell models, which are part of the Flux LoRA DLC Space. If I missed any that are particularly interesting within the Flux group, feel free to contribute to the space!

🥳Flux LoRA Dlc : https://huggingface.co/spaces/prithivMLmods/FLUX-LoRA-DLC

Thankyou!

John6666 victor oceansweep djuna prithivMLmods Ngrthm


CoDA: Instructive Chain-of-Domain Adaptation with Severity-Aware Visual
  Prompt Tuning (2403.17369)

☁☔ New Research Alert! ❄🌙
📄 Title: CoDA: Instructive Chain-of-Domain Adaptation with Severity-Aware Visual Prompt Tuning

📝 Description: CoDA is a UDA methodology that boosts models to understand all adverse scenes (☁,☔,❄,🌙) by highlighting the discrepancies within these scenes. CoDA achieves state-of-the-art performances on widely used benchmarks.

👥 Authors: Ziyang Gong, Fuhao Li, Yupeng Deng, Deblina Bhattacharjee, Xiangwei Zhu, Zhenming Ji

🔗 Paper: https://huggingface.co/papers/2403.17369

📁 Repository: https://github.com/Cuzyoung/CoDA

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🔍 Keywords: #CoDA #DomainAdaptation #VisualPromptTuning #SAVPT #DeepLearning #Innovation

DmitryRyumin Cusyoung reach-vb osanseviero samusenps shiv2050 ssml2050 Makya

DmitryRyumin nikgr reach-vb osanseviero Cusyoung dimbyTa Makya


[Teaser]: We will post the T2I-Scorer series on HF, a sub project of the Q-Future (esp., Q-Align) that provide a family of more accurate evaluators for T2I generation! Stay tuned for our updates!


Are you looking for the perfect leaderboard/arena for your use case? 👀

There's a new tool for this!
https://huggingface.co/spaces/leaderboards/LeaderboardFinder

Select your modality, language, task... then search! 🔍
Some categories of interest:
- does the leaderboard accept submissions?
- is the test set private or public?
- is it using an automatic metric, human evaluators, or llm as a judge?

The spaces list is build from space metadata, and reloaded every hour.

Enjoy!

m-ric nikgr VictorSanh osanseviero bmorphism


Excited to introduce Jamba by AI21
https://huggingface.co/ai21labs/Jamba-v0.1

We are thrilled to announce Jamba, the world’s first production-grade Mamba based model.

Key Features:
- First production-grade Mamba based model built on a novel SSM-Transformer hybrid architecture
- 3X throughput on long contexts compared to Mixtral 8x7B
- Democratizes access to a massive 256K context window
- The only model in its size class that fits up to 140K context on a single GPU

Jamba is based on a novel architecture that combines Mamba and Transformer. While our initial results show great efficiency gains, we expect this to be further explored and improved with the help of the community.

Check out our blog post for more info: https://ai21-labs.webflow.io/blog/announcing-jamba

osanseviero nikgr taufiqdp samusenps ajibawa-2023 Srulikbdd theKibster victor VictorSanh reach-vb apollomarv Asaf-Yehudai MaziyarPanahi DataSoul callmeraxz tatakof

osanseviero samusenps aswathitsme Srulikbdd reach-vb MaziyarPanahi

osanseviero sadafwalliyani reach-vb wenbopan

sadafwalliyani shiv2050 wenbopan DataSoul


Long-form factuality in large language models (2403.18802)

A new paper titled "Long-Form Factuality in Large Language Models" proposes a new approach to evaluate the long-form factuality of large language models using an AI agent! They introduce SAFE (Search-Augmented Factuality Evaluator) which leverages an LLM to break down responses into individual facts, query Google to verify each fact, and perform multi-step reasoning.

Keypoints:
* SAFE (Search-Augmented Factuality Evaluator) is an automated method using an LLM agent to evaluate factuality
* It also introduces LongFact, a 2,280 prompt set spanning 38 topics to test open-domain factual knowledge
* SAFE achieves a 72% humans agreement while being 20x cheaper. It also wins 76% of the disagreements measured on a small scale experiment where a more thorough human procedure (researchers + full internet search) was used.
* Larger models like GPT-4, Claude Opus and Gemini Ultra tend to exhibit better long-form factuality.

Paper: https://huggingface.co/papers/2403.18802
Code and data: https://github.com/google-deepmind/long-form-factuality

Congrats to the authors for their work!


A Little guide to building Large Language Models in 2024

This is a post-recording of a 75min lecture I gave two weeks ago on how to train a LLM from scratch in 2024. I tried to keep it short and comprehensive – focusing on concepts that are crucial for training good LLM but often hidden in tech reports.

In the lecture, I introduce the students to all the important concepts/tools/techniques for training good performance LLM:
* finding, preparing and evaluating web scale data
* understanding model parallelism and efficient training
* fine-tuning/aligning models
* fast inference

There is of course many things and details missing and that I should have added to it, don't hesitate to tell me you're most frustrating omission and I'll add it in a future part. In particular I think I'll add more focus on how to filter topics well and extensively and maybe more practical anecdotes and details.

Now that I recorded it I've been thinking this could be part 1 of a two-parts series with a 2nd fully hands-on video on how to run all these steps with some libraries and recipes we've released recently at HF around LLM training (and could be easily adapted to your other framework anyway):
*`datatrove` for all things web-scale data preparation: https://github.com/huggingface/datatrove
*`nanotron` for lightweight 4D parallelism LLM training: https://github.com/huggingface/nanotron
*`lighteval` for in-training fast parallel LLM evaluations: https://github.com/huggingface/lighteval

Here is the link to watch the lecture on Youtube: https://www.youtube.com/watch?v=2-SPH9hIKT8
And here is the link to the Google slides: https://docs.google.com/presentation/d/1IkzESdOwdmwvPxIELYJi8--K3EZ98_cL6c5ZcLKSyVg/edit#slide=id.p

Enjoy and happy to hear feedback on it and what to add, correct, extend in a second part.

clem osanseviero giux78 samusenps dillfrescott mohammedbriman reach-vb rwightman Venkman42 gusthema 9bow amenur lisuizhe The-Great-Genius danja drFarid MicheleStingo chkla Skier8402 Artples alemiaschi alirezamsh catastropiyush dlicari iproskurina vikas mkhalifa mertege ianlee1996 LiuPengNGP salkhon jdzw2014 serdarcaglar ma7583 derek-thomas fpadovani zackli4ai Davidqian123


𝐓𝐡𝐞 𝐫𝐞𝐭𝐮𝐫𝐧 𝐨𝐟 𝐭𝐡𝐞 𝐑𝐍𝐍𝐬 ⚔ 𝐍𝐞𝐰 𝐌𝐚𝐦𝐛𝐚-𝐛𝐚𝐬𝐞𝐝 𝐚𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 "𝐉𝐚𝐦𝐛𝐚"

Since the release of BERT by Google in 2019, Transformers architecture have taken over machine learning thanks to their 𝗮𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗺𝗲𝗰𝗵𝗮𝗻𝗶𝘀𝗺, that gives them the ability to focus on important points of the input. But 𝙖𝙩𝙩𝙚𝙣𝙩𝙞𝙤𝙣 𝙘𝙤𝙢𝙥𝙪𝙩𝙖𝙩𝙞𝙤𝙣 𝙞𝙨 𝙦𝙪𝙖𝙙𝙧𝙖𝙩𝙞𝙘 𝙞𝙣 𝙩𝙝𝙚 𝙞𝙣𝙥𝙪𝙩 𝙡𝙚𝙣𝙜𝙩𝙝.

💫 The Mamba paper, published in December 2023, announced the return of the RNNs: it has no attention, but integrates a selection mechanism, which should be able to reproduce the “focus” ability of attention, in an architecture for which the compute requirements 𝗴𝗿𝗼𝘄 𝗼𝗻𝗹𝘆 𝗹𝗶𝗻𝗲𝗮𝗿𝗹𝘆 𝗶𝗻 𝗶𝗻𝗽𝘂𝘁 𝗹𝗲𝗻𝗴𝘁𝗵!
🤔 Would this work? We had yet to see a large Mamba model recovering the performance of Attention-based Transformers.

💥 But now it's done! A (Mamba + Transformers) hybrid just beat Transformers!

The AI21 Labs team just released Jamba.
They insert a few Transformer layers to inject some attention in a big pile of Mamba layers, thus getting the best of both worlds.

𝙏𝙇;𝘿𝙍:
🏗 𝗡𝗲𝘄 𝗠𝗼𝗘 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲: 4 Jamba blocks, each of these being 7 Mamba layers for 1 Transformer.
🏋 𝟱𝟮𝗕 𝗽𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿𝘀, 𝟭𝟮𝗕 𝗮𝗰𝘁𝗶𝘃𝗲 𝗮𝘁 𝗶𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲: This reduction is enabled by Mixture of Experts, and similar to Mixtral (47B parameters - 13B active).
🏎 𝗦𝗽𝗲𝗲𝗱: 𝘅𝟯 𝘁𝗵𝗿𝗼𝘂𝗴𝗵𝗽𝘂𝘁. Jamba is much faster than similar-sized Transformer models on long contexts.
📏 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗹𝗲𝗻𝗴𝘁𝗵: 𝟭𝟰𝟬𝗞 𝘁𝗼𝗸𝗲𝗻𝘀 on a single 80GB A100!
💪 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲: 𝘀𝘁𝗮𝘁𝗲-𝗼𝗳-𝘁𝗵𝗲-𝗮𝗿𝘁 𝗳𝗼𝗿 𝘁𝗵𝗶𝘀 𝘀𝗶𝘇𝗲. The small injection of attention seems sufficient since Jamba beats the open-source reference Mixtral-8x7B on many benchmarks!

Try it here 👉 https://huggingface.co/ai21labs/Jamba-v0.1

monsoon-nlp lunarflu carbonox-infernox dhuynh95 osanseviero vonehrenheim nikgr samusenps Amano13 reach-vb Chunte seyf1elislam blanchon


Just now, we release a small MoE model, Qwen1.5-MoE-A2.7B, a 14B model with 2.7B activated parameters. Leaving the hype, I would love to share more things here in HF. But if you don't know much about this, check our blog for more info: https://qwenlm.github.io/blog/qwen-moe/

At the beginning, it was trying with the MoE stuff, making Megatron work well with MegaBlocks. As always, we worked with small ones first. However, we have been struggling with a lot of details.

With megablocks and so many tricks that make training MoE models work, it is almost impossible to fail. The challenge is actually how good your model is. Then things became more complex than I had expected. Finegrained experts actually pissed me off but damn it works for the model at this scale. However, it brings complexity to the model, and this is somehow why at this moment our codes are not merged into llama.cpp cuz it really brings problems. Shared experts might be good, but we need more engineering efforts to really unleash its benefits in inference acceleration.

For the community, this is actually our first time releasing an MoE model. We don't know what will happen to us, but we are prepared for complaints. I just hope that we can really make things clear, and provide a good recipe to play with our MoE model just like people playing with Mixtral.

Nbardy lunarflu CulturedMan osanseviero nikgr manishiitg reach-vb pierrci aguspiza xiaoniqiu alielfilali01 Shuaiii

andysalerno osanseviero Fizzarolli samusenps reach-vb alielfilali01 btcnhung1299 Shuaiii


 🎈 LLM Benchmarks Update!

**tl;dr: do not depend on benchmark leaderboards to choose your "chatbot" model! (Especially for non-English languages.)**

First of all, I'm discontinuing the Open #Dutch #LLM Leaderboard (https://lnkd.in/eFnsaFR6). It will stay online for now, but I urge the use of the ScandEval leaderboard instead (https://scandeval.com/dutch-nlg/) by @saattrupdan. It contains more tasks, has better reproducibility and statistics (CI) and a flexible back-end library (`scandeval`) to run your own benchmarks with. As part of project "Leesplank" (with Michiel Buisman and Maarten Lens-FitzGerald) we recently added GPT-4-1106-preview scores to add a good "target" to the leaderboard.

An important note here is that benchmark leaderboards are not a golden truth. Especially evaluating generative models is hard. You run into issues like prompt engineering (and sensitivity of models to one or other prompt), structured output generation, and - quite simply - "how to automatically evaluate open-ended generation".

💡 Another important but under-discussed facet is the discrepancy between models' capability of understanding vs. generating *in different languages* (so the NLU part of NLG benchmarking). In other words: some of the listed models score really well on, e.g., MCQ benchmarks but are not suitable to use as DUTCH chat bots. Interestingly, some of these models seem to understand questions in Dutch and are able to pick the right answer (because they have good knowledge or reasoning skills), but generating fluent and grammatical Dutch is something else entirely! This is perhaps also true for humans: it's easier to sort-of grasp the meaning of a new language and answer with "Yes" or "No", but answering fluently in the language is much harder! Yet, your language production fluency does not necessarily say anything about your knowledge and reasoning skills.

Hopefully we can get a chat arena for Dutch some day - user feedback is the most powerful metric!

davanstrien lunarflu nikgr saattrupdan reach-vb shiv2050 ssml2050

Rijgersberg osanseviero lunarflu robinsmits reach-vb


Mini-Gemini: Mining the Potential of Multi-modality Vision Language
  Models (2403.18814)

VILA: On Pre-training for Visual Language Models (2312.07533)

FeatUp: A Model-Agnostic Framework for Features at Any Resolution (2403.10516)

Diaries of Open Source. Part 11 🚀

🚀Databricks release DBRX, potentially the best open access model! A 132B Mixture of Experts with 36B active params and trained on 12 trillion tokens
Blog: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm
Base and instruct models: https://hf.co/collections/databricks/dbrx-6601c0852a0cdd3c59f71962
Demo: https://hf.co/spaces/databricks/dbrx-instruct

🤏1-bit and 2-bit quantization exploration using HQQ+
Blog post: https://mobiusml.github.io/1bit_blog/
Models: https://hf.co/collections/mobiuslabsgmbh/llama2-7b-hqq-6604257a96fc8b9c4e13e0fe
GitHub: https://github.com/mobiusml/hqq

📚Cosmopedia: a large-scale synthetic dataset for pre-training - it includes 25 billion tokens and 30 million files
Dataset: https://hf.co/datasets/HuggingFaceTB/cosmopedia
Blog: https://hf.co/blog/cosmopedia

⭐Mini-Gemini: multi-modal VLMs, from 2B to 34B
Models: https://hf.co/collections/YanweiLi/mini-gemini-6603c50b9b43d044171d0854
Paper: https://huggingface.co/papers/2403.18814
GitHub: https://github.com/dvlab-research/MiniGemini

🔥VILA - On Pre-training for VLMs
Models: https://hf.co/collections/Efficient-Large-Model/vila-on-pre-training-for-visual-language-models-65d8022a3a52cd9bcd62698e
Paper: https://hf.co/papers/2312.07533

Misc
👀 FeatUp: a framework for image features at any resolution: https://hf.co/spaces/mhamilton723/FeatUp https://hf.co/papers/2403.10516
🍞ColBERTus Maxiums, a colbertialized embedding model https://hf.co/mixedbread-ai/mxbai-colbert-large-v1
🖌Semantic Palette, a new drawing paradigm  https://hf.co/spaces/ironjr/SemanticPalette
🧑⚕HistoGPT, a vision model that generates accurate pathology reports  https://hf.co/marr-peng-lab/histogpt https://www.medrxiv.org/content/10.1101/2024.03.15.24304211v1

mobicham appoose loubnabnl davanstrien samusenps DmitryRyumin ajibawa-2023 clem GO4code victor vladbogo nikgr VictorSanh reach-vb manuel-tran

ironjr loubnabnl samusenps clem reach-vb manuel-tran


ViTAR: Vision Transformer with Any Resolution (2403.18361)

🎉 🎉 🎉 Happy to share our recent work. We noticed that image resolution plays an important role, either in improving multi-modal large language models (MLLM) performance or in Sora style any resolution encoder decoder, we hope this work can help lift restriction of 224x224 resolution limit in ViT.

https://huggingface.co/papers/2403.18361

ajibawa-2023 samusenps DmitryRyumin clem lunarflu victor nikgr reach-vb shiv2050 ak0601 ncoop57 talaviyabhavik


Compared Effect Of Image Captioning For SDXL Fine-tuning / DreamBooth Training for a Single Person, 10.3 GB VRAM via OneTrainer

Sadly the post character count is limited so please read full info on Medium here

https://medium.com/@furkangozukara/compared-effect-of-image-captioning-for-sdxl-fine-tuning-dreambooth-training-for-a-single-person-961087e42334


AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation (2403.17694)

🚀🎭🌟 New Research Alert! 🌟 🎭🚀
📄 Title: AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation 🔝

📝 Description: AniPortrait is a novel framework for generating photorealistic portrait animations driven by audio and a reference image, with superior facial naturalness, pose variety, and visual quality, with potential applications in facial motion editing and facial reenactment.

👥 Authors: Huawei Wei, @ZJYang, Zhisheng Wang

🔗 Paper: https://huggingface.co/papers/2403.17694

📁 Repository: https://github.com/Zejun-Yang/AniPortrait

🤗 Demo: https://huggingface.co/spaces/ZJYang/AniPortrait_official
🔥 Model 🤖: https://huggingface.co/ZJYang/AniPortrait

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords:  #AniPortrait #Animation #AudioDriven #Photorealistic #FacialAnimation #DeepLearning #Innovation

DmitryRyumin hinach4n samusenps MiSTe-R clem lunarflu victor reach-vb Poulain


The Unreasonable Ineffectiveness of the Deeper Layers (2403.17887)

The Unreasonable Ineffectiveness of the Deeper Layers

https://huggingface.co/papers/2403.17887

We empirically study a simple layer-pruning strategy for popular families of open-weight pretrained LLMs, finding minimal degradation of performance on different question-answering benchmarks until after a large fraction (up to half) of the layers are removed. To prune these models, we identify the optimal block of layers to prune by considering similarity across layers; then, to "heal" the damage, we perform a small amount of finetuning. In particular, we use parameter-efficient finetuning (PEFT) methods, specifically quantization and Low Rank Adapters (QLoRA), such that each of our experiments can be performed on a single A100 GPU. From a practical perspective, these results suggest that layer pruning methods can complement other PEFT strategies to further reduce computational resources of finetuning on the one hand, and can improve the memory and latency of inference on the other hand. From a scientific perspective, the robustness of these LLMs to the deletion of layers implies either that current pretraining methods are not properly leveraging the parameters in the deeper layers of the network or that the shallow layers play a critical role in storing knowledge.

samusenps ajibawa-2023 osanseviero clem ishmnnit Aryanne KvrParaskevi


SegGPT is a vision generalist on image segmentation, quite like GPTs for computer vision ✨
It comes with the last release of transformers 🎁  Demo and more in this post!
SegGPT is an extension of the Painter, where you speak to images with images: the model takes in an image prompt, transformed version of the image prompt, the actual image you want to see the same transform, and expected to output the transformed image.
SegGPT consists of a vanilla ViT with a decoder on top (linear, conv, linear).
The model is trained on diverse segmentation examples, where they provide example image-mask pairs, the actual input to be segmented, and the decoder head learns to reconstruct the mask output.
This generalizes pretty well!
The authors do not claim state-of-the-art results as the model is mainly used zero-shot and few-shot inference. They also do prompt tuning, where they freeze the parameters of the model and only optimize the image tensor (the input context).
Thanks to 🤗 transformers you can use this model easily!
See here https://huggingface.co/docs/transformers/en/model_doc/seggpt
I have built an app for you to try it out. I combined SegGPT with Depth Anything Model, so you don't have to upload image mask prompts in your prompt pair 🤗
Try it here https://huggingface.co/spaces/merve/seggpt-depth-anything
Also check out the collection https://huggingface.co/collections/merve/seggpt-660466a303bc3cd7559d271b

samusenps osanseviero clem lunarflu talaviyabhavik


Thanks to the 4000+ attempts to jailbreak LLM chatbots and 2000+ votes cast on the Chatbot Guardrails Arena in the 4 days since release!

Based on the players' feedback, we have updated the instructions to be clearer and to emphasize that players should vote after identifying a secure chatbot.

Tell us what you think when you play again: https://huggingface.co/spaces/lighthouzai/guardrails-arena

samusenps osanseviero Tom-Neverwinter clefourrier clem lunarflu reach-vb AtAndDev normanschizogh chuangxinlezhi sbrandeis

samusenps osanseviero Tom-Neverwinter clem lunarflu reach-vb AtAndDev

samusenps Tom-Neverwinter clem lunarflu reach-vb AtAndDev xpgx1


Here, we share the technical details of InternLM2, the current state-of-the-art open-source LLMs!!

See collections https://huggingface.co/collections/internlm/internlm2-65b0ce04970888799707893c
Paper: https://huggingface.co/papers/2403.17297

samusenps osanseviero clem bedirt victor


Have we really squeezed out the capacity of a compact chat model? Thrilled to see our latest open model, Starling-7B, ranks 13th among all models in Chatbot Arena!
🚀 As a 7B model, Starling surpasses larger open and proprietary models, including Claude-2, GPT-3.5-Turbo, Gemini Pro, Mixtral 8x7B and Llama2-70B, and is currently the best 7B chat model in Chatbot Arena!
Try out the model on HF here: https://huggingface.co/Nexusflow/Starling-LM-7B-beta

brian-yu-nexusflow banghua clem ThWu jian-zhang-nexusflow mahiatlinux osanseviero unnaymed Lewdiculous dalraf lunarflu mohammedbriman ucyang

banghua brian-yu-nexusflow damon-mosk-aoyama-nexusflow clem ThWu jian-zhang-nexusflow mahiatlinux coolkrishds osanseviero natolambert lunarflu

banghua brian-yu-nexusflow samusenps clem ThWu jian-zhang-nexusflow mahiatlinux osanseviero dalraf lunarflu

brian-yu-nexusflow banghua ThWu jian-zhang-nexusflow mahiatlinux lunarflu

banghua ThWu jian-zhang-nexusflow mahiatlinux lunarflu brian-yu-nexusflow


New state-of-the-art open LLM! 🚀 Databricks just released DBRX, a 132B MoE trained on 12T tokens. Claiming to surpass OpenAI GPT-3.5 and is competitive with Google Gemini 1.0 Pro. 🤯

TL;DR
🧮 132B MoE with 16 experts with 4 active in generation
🪟 32 000 context window
📈 Outperforms open LLMs on common benchmarks, including MMLU
🚀 Up to 2x faster inference than Llama 2 70B
💻 Trained on 12T tokens
🔡 Uses the GPT-4 tokenizer
📜 Custom License, commercially useable

Collection: https://huggingface.co/collections/databricks/dbrx-6601c0852a0cdd3c59f71962
Demo: https://huggingface.co/spaces/databricks/dbrx-instruct

Kudos to the Team at Databricks and MosaicML for this strong release in the open community! 🤗

victor alielfilali01 Shinku osanseviero samusenps giux78 Tom-Neverwinter mmhamdy sepal giulianobr carlos-eduardo eppie2 VictorSanh dalraf lunarflu thomwolf clem KvrParaskevi dagelf flymonk

clem Tom-Neverwinter mmhamdy eppie2 dalraf lunarflu mohammedbriman matthewgjohnson


 🐰 Happy ML Easter! 🐰

I recently had a wager with my colleagues which had me create AI-assisted videos of myself in an Easter Bunny costume singing an AI-generated easter song (in different languages).

I did it in 3 simple steps:
🐇I created an easter Bunny version of myself with this space: https://huggingface.co/spaces/multimodalart/Ip-Adapter-FaceID by @multimodalart
🎶 Created a song on suno.ai
🎞 Used the Dreamtalk space by @fffiloni to animate the image, conditioned on the song: https://huggingface.co/spaces/fffiloni/dreamtalk

fffiloni osanseviero clem victor linoyts Wauplin kramp yyassif MarinaraSpaghetti sayandafadar alielfilali01 sbrandeis


Would 1-2 sentence tl;dr summaries of datasets on the Hub be useful for you?

For example, for the https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T dataset, would the following summary help give you a quick sense of its content?

> tl;dr: RedPajama is a fully open-source implementation of the LLaMa dataset, consisting of 1.2 trillion tokens from sources like Commoncrawl, C4, GitHub, Books, ArXiv, Wikipedia, and StackExchange, primarily in English, and is structured with metadata for each text sample.

I've created a dataset with example summaries of the 500 most liked datasets on the Hub: https://huggingface.co/datasets/davanstrien/dataset-tldr

Would these kinds of summaries be helpful?

osanseviero merve clem samusenps sequelbox DataSoul Arakinas


Major TOM: Expandable Datasets for Earth Observation (2402.12095)

𝗠𝗮𝗷𝗼𝗿 𝗧𝗢𝗠: 𝗣𝗹𝗮𝗻𝗲𝘁 𝗘𝗮𝗿𝘁𝗵 𝗶𝘀 𝗯̶𝗹̶𝘂̶𝗲̶ 𝟱.𝟰𝟬𝟱 𝗚𝗛𝘇

🚨 EXPANSION RELEASE: 𝗦𝗲𝗻𝘁𝗶𝗻𝗲𝗹-𝟭 𝗶𝘀 𝗻𝗼𝘄 𝗮𝘃𝗮𝗶𝗹𝗮𝗯𝗹𝗲 in the MajorTOM-Core!
https://huggingface.co/datasets/Major-TOM/Core-S1RTC

🏎 Together with @aliFrancis we've been racing to release the first official expansion to the Major TOM project.

MajorTOM-Core-S1RTC contains 1,469,955 of SAR images paired to Sentinel-2 images from Core-S2.

🔍We cover more than 65% of the optical coverage with an average time shift of 7 days.

16 TB of radiometrically calibrated SAR imagery, available in the exact same format as the existing Major-TOM data.

🗺 You can explore instantly in our viewing app:
https://huggingface.co/spaces/Major-TOM/MajorTOM-Core-Viewer

So, what now?

🧱 𝐂𝐨𝐦𝐦𝐮𝐧𝐢𝐭𝐲 𝐆𝐫𝐨𝐰𝐭𝐡: our community continues to grow! To coordinate the upcoming expansions as well as use cases of the open data, we will organise a meet up on 23 April, you can 𝐫𝐞𝐠𝐢𝐬𝐭𝐞𝐫 𝐲𝐨𝐮𝐫 𝐢𝐧𝐭𝐞𝐫𝐞𝐬𝐭 here: https://forms.gle/eBj8JvibJx9b6PLf9

🚂 𝐎𝐩𝐞𝐧 𝐃𝐚𝐭𝐚 𝐟𝐨𝐫 𝐎𝐩𝐞𝐧 𝐌𝐨𝐝𝐞𝐥𝐬: Major-TOM Core dataset is currently supporting several strands of ongoing research within and outwith our lab and we are looking forward to the time when we can release models that take advantage of that data! https://huggingface.co/Major-TOM

📌 𝐏𝐨𝐬𝐭𝐞𝐫 𝐚𝐭 𝐈𝐆𝐀𝐑𝐒𝐒: We will present Major TOM project as a poster at IGARSS in Athens (July) - come talk to us if you're there! You can access the paper here: https://huggingface.co/papers/2402.12095

🌌 Developed at European Space Agency Φ-lab in partnership with Hugging Face

aliFrancis davanstrien osanseviero clem Tom-Neverwinter robmarkcole


Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding
  Model Mechanisms (2403.17806)

🔍 Today's pick in Interpretability & Analysis of LMs: Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms by @mwhanna @sandropezzelle @belinkov

Edge attribution patching (EAP) is a circuit discovery technique using gradients to approximate the effects of causal intervening on each model edge. In the literature, its effectiveness is validated by comparing the overlap of its resulting circuits with those found via causal interventions (much more expensive).

This work:

1. Proposes a new method for faithful and efficient circuit discovery named edge attribution patching with integrated gradients (EAP-IG)
2. Evaluates the faithfulness EAP, EAP-IG and activation patching, i.e. whether behavior of the model remains consistent after all non-circuit edges are ablated.
3. Highlights that, while the no-overlap and full-overlap of EAP-like methods with activation patching results are generally good indicators of unfaithful and faithful (respectively) circuit identification, circuits with moderate overlap cannot generally assumed to be faithful to model behavior.

An advantage of EAP-IG is enabling the usage of KL-Divergence as a target for gradient propagation, which is not possible in the case of raw gradient-based EAP.

EAP-IG runtime is approximately similar to the one of EAP, with a small number of steps to approximate the gradient integral.

Importantly, circuit faithfulness does not imply completeness, i.e. whether all components participating towards a specific task were accounted for. This aspect is identified as interesting for future work.

📄 Paper: https://huggingface.co/papers/2403.17806

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9

javifer osanseviero samusenps clem giux78 mmhamdy alemiaschi


Based on the work of @mrinaldi and @ruggsea  we just released the biggest - ready for training - conversational dataset based on Usenet data in the Italian language 🇮🇹🇮🇹🇮🇹🇮🇹🇮🇹🇮🇹🇮🇹.   It contains about 9 millions of conversations made by real humans.

https://huggingface.co/datasets/mii-community/UsenetArchiveIT-conversations

mrinaldi gsarti fgamezf osanseviero giux78 merve clem

gsarti osanseviero merve samusenps clem giux78


Exciting news! 🎉 I've created the OpenCerebrum datasets, open-source alternatives to Aether Research's proprietary Cerebrum dataset.

The first, OpenCerebrum SFT, is a text-generation and question-answering dataset with ~1.2M examples, curated from sources like Open-Orca, glaiveai, camel-ai, and more! 📚

The second, OpenCerebrum DPO, is a smaller dataset with ~21k examples, focusing on data point optimization. It's curated from sources like jondurbin, argilla, grimulkan, and others. 📊

Both datasets are licensed under Apache-2.0 and are available in English. They're ready for use in your projects, and I welcome any feedback for future improvements! 🚀

https://huggingface.co/datasets/Locutusque/OpenCerebrum-dpo
https://huggingface.co/datasets/Locutusque/OpenCerebrum-SFT
https://huggingface.co/Locutusque/OpenCerebrum-1.0-7b-SFT
https://huggingface.co/Locutusque/OpenCerebrum-1.0-7b-DPO

afrideva samusenps ajibawa-2023 giux78 osanseviero lewtun dvilasuero lhoestq victor Tom-Neverwinter InferenceIllusionist MexIvanov clefourrier mlabonne

monsoon-nlp Xaln osanseviero lewtun dvilasuero Tom-Neverwinter shiv2050

samusenps osanseviero lewtun dvilasuero Tom-Neverwinter


Current LLMs are very susceptible to generating toxic, harmful and even dangerous content. They can also generate outputs with gender or racial biases.

The Biden-Harris Executive Order (https://www.federalregister.gov/documents/2023/11/01/2023-24283/safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence) sets forth guidelines on what is considered a safe AI system.

Following up on these guidelines, we present the world's first open source Biden-Harris Executive Order Red teamed Multilingual Language Model: Aurora-M.

The model is trained on 5 languages: English, Hindi, Japanese, Vietnamese and Finnish.

Blog: https://huggingface.co/blog/mayank-mishra/aurora
Paper coming out soon.

Base model: https://huggingface.co/aurora-m/aurora-m-base (not safety tuned)
Instruct model: https://huggingface.co/aurora-m/aurora-m-instruct (not safety tuned)
Red teamed model: https://huggingface.co/aurora-m/aurora-m-biden-harris-redteamed (safety tuned according to the order mentioned above)

samusenps oneiroid clem lunarflu antiven0m adamelliotfields Joseph717171

samusenps mayank-mishra clem diwank lunarflu sted97


Visual CoT: Unleashing Chain-of-Thought Reasoning in Multi-Modal
  Language Models (2403.16999)

A new paper introduces Visual CoT, a new approach that enhances multi-modal large language models with visual chain-of-thought reasoning capabilities. This allows language models to dynamically identify and focus on specific regions within images that are most relevant for answering questions, mimicking human-like efficient visual reasoning.

Keypoints:
* Introduces the 373k Visual CoT dataset with bounding box annotations highlighting essential image regions
* Proposes a multi-turn pipeline for focusing on relevant visual inputs
* Achieves strong results on multi-modal benchmarks

Paper: https://huggingface.co/papers/2403.16999
Code, data and other resources: https://github.com/deepcs233/Visual-CoT

Congrats to the authors for their work!

samusenps Csplk osanseviero thesouthfrog Deping sbrandeis


𝐒𝐨 𝐡𝐨𝐰 𝐢𝐬 𝐭𝐡𝐞 𝐧𝐞𝐱𝐭 𝐭𝐨𝐤𝐞𝐧 𝐬𝐞𝐥𝐞𝐜𝐭𝐞𝐝?

𝐒𝐨 𝐡𝐨𝐰 𝐢𝐬 𝐭𝐡𝐞 𝐧𝐞𝐱𝐭 𝐭𝐨𝐤𝐞𝐧 𝐬𝐞𝐥𝐞𝐜𝐭𝐞𝐝?

𝙏𝙧𝙮 𝙞𝙩 𝙝𝙚𝙧𝙚:

𝙏𝙧𝙮 𝙞𝙩 𝙝𝙚𝙧𝙚:

𝗛𝗼𝘄 𝗱𝗼𝗲𝘀 𝗯𝗲𝗮𝗺 𝘀𝗲𝗮𝗿𝗰𝗵 𝗱𝗲𝗰𝗼𝗱𝗶𝗻𝗴 𝘄𝗼𝗿𝗸? ➡ 𝙉𝙚𝙬 𝙫𝙞𝙨𝙪𝙖𝙡𝙞𝙯𝙖𝙩𝙞𝙤𝙣 𝙩𝙤𝙤𝙡! 👀

In Decoder-type LLMs like GPT4 or Mistral-Large, the output is generated one token (=word part) at a time. That's why they're nicknamed "stochastic parrots": the "thinking" process only happens one step at a time, so it can seem really myopic.

𝐒𝐨 𝐡𝐨𝐰 𝐢𝐬 𝐭𝐡𝐞 𝐧𝐞𝐱𝐭 𝐭𝐨𝐤𝐞𝐧 𝐬𝐞𝐥𝐞𝐜𝐭𝐞𝐝?

📊 Given its input sentence like "𝘞𝘩𝘢𝘵 𝘪𝘴 𝘵𝘩𝘦 7𝘵𝘩 𝘍𝘪𝘣𝘰𝘯𝘢𝘤𝘤𝘪 𝘯𝘶𝘮𝘣𝘦𝘳? 𝘛𝘩𝘦 7𝘵𝘩 𝘍𝘪𝘣𝘰𝘯𝘢𝘤𝘤𝘪 𝘯𝘶𝘮𝘣𝘦𝘳", the Decoder LLM generates, for each token in its vocabulary, a score that represents this token's probability of coming next.
For instance: "𝙞𝙨" gets score 0.56, and "𝙘𝙖𝙣" gets score 0.35.

🤑 𝐆𝐫𝐞𝐞𝐝𝐲 𝐝𝐞𝐜𝐨𝐝𝐢𝐧𝐠 is the naive option where you simply take the next most probable token at each step. But this creates paths that maximize very short-term rewards, thus may overlook better paths for the long term (like this time when you played FIFA all evening and arrived unprepared to your school exam on the next day).
In our example, the next highest score token might be "𝙞𝙨", but this will strongly bias the LLM towards giving an hasty response. On the opposite, starting with "𝙘𝙖𝙣" could have been completed with "𝘣𝘦 𝘰𝘣𝘵𝘢𝘪𝘯𝘦𝘥 𝘧𝘳𝘰𝘮 𝘤𝘰𝘮𝘱𝘶𝘵𝘪𝘯𝘨 𝘱𝘳𝘦𝘷𝘪𝘰𝘶𝘴 𝘍𝘪𝘣𝘰𝘯𝘢𝘤𝘤𝘪 𝘯𝘶𝘮𝘣𝘦𝘳𝘴 𝘧𝘪𝘳𝘴𝘵", which steers the LLM towards a correct reasoning!

🗺 𝐁𝐞𝐚𝐦 𝐬𝐞𝐚𝐫𝐜𝐡 improves on greedy decoding by generating at each step several paths - called beams - instead of one. This allows the generation to explore a much larger space, thus find better completions. In our example, both the "𝙞𝙨" and the "𝙘𝙖𝙣" completion could be tested. ✅

👉 I've created a tool to let you visualize it, thank you @joaogante for your great help!
𝙏𝙧𝙮 𝙞𝙩 𝙝𝙚𝙧𝙚: https://huggingface.co/spaces/m-ric/beam_search_visualizer

samusenps matlok joaogante clefourrier fieryTransition Jafta awagner-mainz

osanseviero Awal mmhamdy samusenps joaogante clefourrier


Very glad to welcome @josefprusa, pioneer of 3D printing and open source hardware, founder of https://www.prusa3d.com/, to the HF Hub 👋

AI applied to 3D printing could be big.

victor London12345 satpalsr diwank ZennyKenny radames adamelliotfields VictorSanh mmhamdy samusenps pcuenq kramp osanseviero stattmone mdouglas clem Skier8402 jarvisx17

Aurelien-Morgan imomayiz mmhamdy samusenps clem


LLM Agent Operating System

https://huggingface.co/papers/2403.16971

The integration and deployment of large language model (LLM)-based intelligent agents have been fraught with challenges that compromise their efficiency and efficacy. Among these issues are sub-optimal scheduling and resource allocation of agent requests over the LLM, the difficulties in maintaining context during interactions between agent and LLM, and the complexities inherent in integrating heterogeneous agents with different capabilities and specializations. The rapid increase of agent quantity and complexity further exacerbates these issues, often leading to bottlenecks and sub-optimal utilization of resources. Inspired by these challenges, this paper presents AIOS, an LLM agent operating system, which embeds large language model into operating systems (OS). Specifically, AIOS is designed to optimize resource allocation, facilitate context switch across agents, enable concurrent execution of agents, provide tool service for agents, and maintain access control for agents. We present the architecture of such an operating system, outline the core challenges it aims to resolve, and provide the basic design and implementation of the AIOS. Our experiments on concurrent execution of multiple agents demonstrate the reliability and efficiency of our AIOS modules. Through this, we aim to not only improve the performance and efficiency of LLM agents but also to pioneer for better development and deployment of the AIOS ecosystem in the future.

samusenps QiushiSun osanseviero alielfilali01 AtAndDev Jakaline


MLX RAG with GGUF Models
Minimal, clean code implementation of RAG with mlx inferencing for GGUF models.

Code: https://github.com/Jaykef/mlx-rag-gguf

The code here builds on vegaluisjose's example, it has been optimized to support RAG-based inferencing for .gguf models. I am using BAAI/bge-small-en for the embedding model, tinyllama-1.1b-chat-v1.0.Q4_0.gguf as base model and the custom vector database script for indexing texts in a pdf file. Inference speeds can go up to ~413 tokens/sec for prompts and ~36 tokens/sec for generation on my M2 Air.

Queries make use of both .gguf (base model) and .npz (retrieval model) simultaneouly resulting in much higher inferencing speeds.


Repurposing Diffusion-Based Image Generators for Monocular Depth
  Estimation (2312.02145)

Quiet-STaR: Language Models Can Teach Themselves to Think Before
  Speaking (2403.09629)

Unlocking the conversion of Web Screenshots into HTML Code with the
  WebSight Dataset (2403.09029)

Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for
  Large Language Models (2403.12881)

Diaries of Open Source. Part 10 🚀

🌼Marigold-LCM: A super fast SOTA Depth Estimator
Demo: https://hf.co/spaces/prs-eth/marigold-lcm
Original paper: https://hf.co/papers/2312.02145
Model: https://hf.co/prs-eth/marigold-lcm-v1-0

🌟Quiet-STaR: A self-teaching technique via internal monologue
Paper: https://hf.co/papers/2403.09629
GitHub: https://github.com/ezelikman/quiet-star
Tweetutorial: https://twitter.com/ericzelikman/status/1768663835106513041

🖼 WebSight v0.2: A image-to-code dataset containing tailwind CSS, images in screenshots, and more!
Dataset: https://hf.co/datasets/HuggingFaceM4/WebSight
Paper: https://hf.co/papers/2403.09029
Blog: https://hf.co/blog/websight

🕵Agent-FLAN - effective agent tuning for LLMs
Paper: https://hf.co/papers/2403.12881
Model: https://hf.co/internlm/Agent-FLAN-7b
Dataset: https://hf.co/datasets/internlm/Agent-FLAN
Website: https://internlm.github.io/Agent-FLAN/

🔥HPT, a family of multimodal LLMs from HyperGAI
Blog post: https://hypergai.com/blog/introducing-hpt-a-family-of-leading-multimodal-llms
Model: https://huggingface.co/HyperGAI/HPT
GitHub: https://github.com/hyperGAI/HPT

🌏Models and datasets around the world
- Tess-70B, a MiQu-70B fine-tune with high-quality data https://hf.co/migtissera/Tess-70B-v1.6
- UNI, a model trained on 100 million pathology images from 100k+ slides https://hf.co/MahmoodLab/UNI
- CONCH, a VLM trained on 1.17 million pathology image-text pairs https://hf.co/MahmoodLab/CONCH

Bingxin toshas DmitryRyumin thomwolf migtissera julien-c London12345 mmhamdy samusenps pcuenq victor sbrandeis

toshas migtissera julien-c mmhamdy pabloce samusenps pcuenq


FlashFace: Human Image Personalization with High-fidelity Identity
  Preservation (2403.17008)

🚀🎭🌟 New Research Alert! 🌟 🎭🚀
📄 Title: FlashFace: Human Image Personalization with High-fidelity Identity Preservation 🔝

📝 Description: FlashFace is a personalized photo editing tool that focuses on high-fidelity identity preservation and improved compliance through advanced encoding and integration strategies.

👥 Authors: Shilong Zhang, Lianghua Huang, @xichenhku et al.

🔗 Paper: https://huggingface.co/papers/2403.17008

🌐 Github Page: https://jshilong.github.io/flashface-page

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #FlashFace #Personalization #HighFidelityIdentity  #DeepLearning #Innovation

DmitryRyumin waqas700297 kgourgou London12345 samusenps KvrParaskevi whooray

dashfunnydashdash loveisp AtonMountlook diwank


Now You Can Full Fine Tune / DreamBooth Stable Diffusion XL (SDXL) with only 10.3 GB VRAM via OneTrainer — Both U-NET and Text Encoder 1 is trained — Compared 14 GB config vs slower 10.3 GB Config

Full config and instructions are shared here : https://www.patreon.com/posts/96028218

Used SG161222/RealVisXL_V4.0 as a base model and OneTrainer to train on Windows 10 : https://github.com/Nerogar/OneTrainer

The posted example x/y/z checkpoint comparison images are not cherry picked. So I can get perfect images with multiple tries.

Trained 150 epochs, 15 images and used my ground truth 5200 regularization images : https://www.patreon.com/posts/massive-4k-woman-87700469

In each epoch only 15 of regularization images used to make DreamBooth training affect

As a caption only “ohwx man” is used, for regularization images just “man”
You can download configs and full instructions here : https://www.patreon.com/posts/96028218

Hopefully full public tutorial coming within 2 weeks. I will show all configuration as well

The tutorial will be on our channel : https://www.youtube.com/SECourses
Training speeds are as below thus durations:

RTX 3060 — slow preset : 3.72 second / it thus 15 train images 150 epoch  2 (reg images concept) : 4500 steps = 4500  3.72 / 3600 = 4.6 hours

RTX 3090 TI — slow preset : 1.58 second / it thus : 4500 * 1.58 / 3600 = 2 hours

RTX 3090 TI — fast preset : 1.45 second / it thus : 4500 * 1.45 / 3600 = 1.8 hours

A quick tutorial for how to use concepts in OneTrainer : https://youtu.be/yPOadldf6bI


The AI Comic Factory 1.2 has been released a few hours ago!

You can now tell the LLM in advance how many pages you will want, for richer and deeper stories

https://www.loom.com/share/b8b2d55fc60249a78df60adaa2673d2f

I the future I would like to improve how layouts are attributed to pages ( seeing the same layout 6x can be a bit boring)

Also, I'm still waiting for some new improved AI models to drop, so stay tuned!

osanseviero victor halbarba musfiqdehan szh clem samusenps ajibawa-2023 radames kramp julien-c Krass1 kuramacn Handoman tschacht Chris chitrark

thebryanalvarado monsoon-nlp clem waqas700297 julien-c Krass1 kuramacn


Introducing Marigold-LCM 🌼 — a FAST version of the now popular state-of-the-art depth estimator! Thanks to the latent consistency distillation, it retains the precision of the original Marigold but reaches the solution in just a few steps!

Check out the teaser video attached below and play with the new demo - it accepts videos now! Also, meet the new team member: Tianfu Wang (@Tianfwang)

🤗 Demo: https://huggingface.co/spaces/prs-eth/marigold-lcm
🤗 Model: https://huggingface.co/prs-eth/marigold-lcm-v1-0
🤗 Original Marigold post: https://huggingface.co/posts/toshas/656973498012745
🤗 Paper: https://huggingface.co/papers/2312.02145
🌐 Website: https://marigoldmonodepth.github.io
👾 Code: https://github.com/prs-eth/marigold
👾 Code: pip install diffusers

DmitryRyumin samusenps osanseviero clem radames Veneco


🚀 Just released version 0.22.0 of the `huggingface_hub` Python library!

Exciting updates include:
✨ Chat-completion API in the InferenceClient!
🤖 Official inference types in InferenceClient!
🧩 Better config and tags in `ModelHubMixin`!
🏆 Generate model cards for your `ModelHubMixin` integrations!
🏎  x3 download speed in `HfFileSystem`!!

Check out the full release notes for more details: https://huggingface.co/spaces/Wauplin/huggingface_hub/discussions/5 👀

pierrci raucheacho cbensimon clefourrier victor trysem nielsr monsoon-nlp samusenps osanseviero mmhamdy clem minhdang ajibawa-2023 julien-c Tonic wizardhat alielfilali01 kramp not-lain gorodecki sbrandeis msheykhmousa


Just released a notebook showing how to finetune moondream: https://github.com/vikhyat/moondream/blob/main/notebooks/Finetuning.ipynb

NickyNicky samusenps peaceAsh musfiqdehan Norod78 osanseviero ajibawa-2023 Tom-Neverwinter clem abdullah damerajee t1u1 notune not-lain antiven0m radames gabrielchua M4tthias Essa20001

kristaller486 Tom-Neverwinter clem julien-c not-lain francedot radames

avinash02 Tom-Neverwinter clem Ksgk-fy not-lain radames


Chronos: Learning the Language of Time Series (2403.07815)

ORPO: Monolithic Preference Optimization without Reference Model (2403.07691)

Diaries of Open Source. Part 9!

⏰Amazon releases Chronos, a family of models for time series
Base model: https://hf.co/amazon/chronos-t5-large
Paper: https://hf.co/papers/2403.07815
Models: https://huggingface.co/collections/amazon/chronos-models-65f1791d630a8d57cb718444

💡ORPO Alignment: align without a reference model nor SFT!
Paper: https://hf.co/papers/2403.07691
Models: https://hf.co/collections/kaist-ai/orpo-65efef87544ba100aef30013
GitHub: https://github.com/xfactlab/orpo

🇺🇳Cohere releases 250M Wikipedia Embeddings in 300+ languages
Data: https://hf.co/datasets/Cohere/wikipedia-2023-11-embed-multilingual-v3
Announcement: https://twitter.com/Nils_Reimers/status/1767891859207057618

🧬SegmentNT: a LLM for annotating DNA at single nucleotide resolution
Models: https://huggingface.co/collections/InstaDeepAI/segmentnt-65eb4941c57808b4a3fe1319
GitHub repo: https://github.com/instadeepai/nucleotide-transformer
Paper: https://www.biorxiv.org/content/10.1101/2024.03.14.584712v1

🚀DynamiCrafter: video generation models for interpolation and looping are out!
Project page: https://doubiiu.github.io/projects/DynamiCrafter/
GitHub: https://github.com/Doubiiu/DynamiCrafter
Demo: https://hf.co/spaces/Doubiiu/DynamiCrafter_interp_loop

🚀Stanford releases Anticipatory Music Transformer:
GitHub: https://github.com/jthickstun/anticipation/
Models: https://hf.co/stanford-crfm
Original blog announcement: https://crfm.stanford.edu/2023/06/16/anticipatory-music-transformer.html

YaTharThShaRma999 giux78 DmitryRyumin mmhamdy samusenps clem


I and @FinancialSupport just release the new Italian Leaderboard:

https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard

It is based on lm-evaluation-harness and at the moment , mainly, on 7 billion models. In the next weeks we will add more models. If you have suggestion or need explanations join our community discord https://discord.gg/a26cRkBCNH


Text2Avatar: Text to 3D Human Avatar Generation with Codebook-Driven
  Body Controllable Attribute (2401.00711)

🚀💃🌟 New Research Alert - ICASSP 2024! 🌟 🕺🚀
📄 Title: Text2Avatar: Text to 3D Human Avatar Generation with Codebook-Driven Body Controllable Attribute 🌟

📝 Description: Text2Avatar is a novel approach that can generate realistic 3D human avatars directly from textual descriptions, enabling multi-attribute control and realistic styling, overcoming the challenges of feature coupling and data scarcity in this domain.

👥 Authors: Chaoqun Gong et al.

📅 Conference: ICASSP, 14-19 April 2024 | Seoul, Korea 🇰🇷

🔗 Paper: https://huggingface.co/papers/2401.00711

🌐 Github Page: https://iecqgong.github.io/text2avatar/

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

📁 Added to the ICASSP-2023-24-Papers: https://github.com/DmitryRyumin/ICASSP-2023-24-Papers

🔍 Keywords: #AvatarGeneration #Text2Avatar #ICASSP2024 #DeepLearning #Innovation

DmitryRyumin samusenps xenagarage Dlbk palloo victor Tom-Neverwinter

palloo GaneshMystic clem Tom-Neverwinter


CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction
  Model (2403.05034)

From Scarcity to Efficiency: Improving CLIP Training via Visual-enriched
  Captions (2310.07699)

MOFI: Learning Image Representations from Noisy Entity Annotated Images (2306.07952)

Efficiency at Scale: Investigating the Performance of Diminutive
  Language Models in Clinical Tasks (2402.10597)

Diaries of Open Source. Part 8!

🤯CRM: Image-to-3D Textured Mesh
Demo: https://hf.co/spaces/Zhengyi/CRM
Model: https://hf.co/Zhengyi/CRM
Project page: https://ml.cs.tsinghua.edu.cn/~zhengyi/CRM/
Paper: https://huggingface.co/papers/2403.05034

🤏Half Quadratic Quantization: super-fast quantization of very large models
Blog post: https://mobiusml.github.io/hqq_blog/
Colab: https://colab.research.google.com/drive/1cG_5R_u9q53Uond7F0JEdliwvoeeaXVN?usp=sharing
Repo: https://github.com/mobiusml/hqq

🤗GemMoE -Gemma + MoE
Model: https://hf.co/Crystalcareai/GemMoE-Base-Random
Collection: https://huggingface.co/collections/Crystalcareai/gemmoe-65f11f4922af97ebe9943591

👀VeCLIP and MOFI, new 0-shot and image retrieval models by Apple, are now open-source!
GitHub: https://github.com/apple/ml-veclip/ and https://github.com/apple/ml-mofi
VeCLIP paper: https://hf.co/papers/2310.07699
MOFI paper: https://hf.co/papers/2306.07952

⚡SPIN: Recipe for alignment with very little data
Collection: https://hf.co/collections/argilla/dibt-prompt-collective-spin-65ef59062518776024395fc3
Tweetutorial: https://twitter.com/argilla_io/status/1767608154697699455

👀ViT Prisma - an interoperability library for vision models
GitHub: https://github.com/soniajoseph/ViT-Prisma

☕OpenLRM: full model and training code are open-sourced
Codebase: https://github.com/3DTopia/OpenLRM
Demo: https://hf.co/spaces/zxhezexin/OpenLRM
Models: https://huggingface.co/zxhezexin

⚗Oxford releases  an extensive PEFT evaluation for bio models
Model: https://hf.co/NTaylor/bio-mobilebert-mimic-mp-lora
GitHub: https://github.com/nlpie-research/efficient-ml
Paper: https://hf.co/papers/2402.10597

🌍Data and models around the world
Hermes 2 Pro 7B: an upgraded Nous Hermes 2 model with strong function calling and JSON capabilities  https://hf.co/NousResearch/Hermes-2-Pro-Mistral-7B
 Navarasa-2.0 : Gemma fine-tuned in 15 indian language https://hf.co/collections/Telugu-LLM-Labs/navarasa-65f5e6ffdf29f02c6d7767ce

dvilasuero DmitryRyumin AugustYuYang samusenps ajibawa-2023 DonAmit197 Zhengyi victor clem radames


Does anyone have experience with finetuning Gemma? Even the 2B variant feels more memory heavy than mistral 7B. I know that its vocabulary is much larger (250k) but I'm a bit surprised that the max batch size that I can get in an A100 80GB is only 2 whereas I could fit 4 with mistral 7B - even though Gemma is much smaller except for the embedding layer. Both runs were using FA, same sequence length, same deepspeed zero 3 settings. Oh and yes I'm using the most recent hot fix of transformers that solves a memory issue with Gemma and others.

Any prior experience that you can share or suggestions to improve throughout?

KnutJaegersberg victor DonAmit197 svernek FlipTip soates clem Andcircle

osanseviero samusenps DonAmit197 svernek clem dillfrescott


Today we launch our space in colab with @dvilasuero  & @davanstrien  so you can help us translate/correct our curated prompt dataset, that will be used to evaluate the performance of Arabic LLMs later and help our community to identify how open models perform on Arabic.

How to Get Involved?

1. Visit our Argilla Space and start reviewing prompts.
https://2a2i-prompt-translation-for-arabic.hf.space/

2. Join our Discord channel in the HuggingFace's discord server to connect with the community and share your insights.
https://discord.com/channels/879548962464493619/1217179730869096448

medmac01 dvilasuero samusenps ZennyKenny SudhaSG osanseviero monsoon-nlp davanstrien clefourrier not-lain KBayoud clem

medmac01 dvilasuero osanseviero davanstrien not-lain KBayoud clem


MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual
  Math Problems? (2403.14624)

MathVerse

Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?

https://huggingface.co/papers/2403.14624

The remarkable progress of Multi-modal Large Language Models (MLLMs) has garnered unparalleled attention, due to their superior performance in visual contexts. However, their capabilities in visual math problem-solving remain insufficiently evaluated and understood. We investigate current benchmarks to incorporate excessive visual content within textual questions, which potentially assist MLLMs in deducing answers without truly interpreting the input diagrams. To this end, we introduce MathVerse, an all-around visual math benchmark designed for an equitable and in-depth evaluation of MLLMs. We meticulously collect 2,612 high-quality, multi-subject math problems with diagrams from publicly available sources. Each problem is then transformed by human annotators into six distinct versions, each offering varying degrees of information content in multi-modality, contributing to 15K test samples in total. This approach allows MathVerse to comprehensively assess whether and how much MLLMs can truly understand the visual diagrams for mathematical reasoning. In addition, we propose a Chain-of-Thought (CoT) evaluation strategy for a fine-grained assessment of the output answers. Rather than naively judging True or False, we employ GPT-4(V) to adaptively extract crucial reasoning steps, and then score each step with detailed error analysis, which can reveal the intermediate CoT reasoning quality by MLLMs. We hope the MathVerse benchmark may provide unique insights to guide the future development of MLLMs.

samusenps AtAndDev shvhio jncarlo thomwolf


GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians (2312.02069)

🚀🎭🌟 New Research Alert - CVPR 2024! 🌟 🎭🚀
📄 Title: GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians 🔝

📝 Description: GaussianAvatars proposes a novel method for creating photorealistic and fully controllable head avatars by combining a parametric morphable face model with a dynamic 3D representation based on rigged 3D Gaussian splats, enabling high-quality rendering and precise animation control.

👥 Authors: Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, Matthias Nießner

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2312.02069

🌐 Github Page: https://shenhanqian.github.io/gaussian-avatars
📁 Repository: https://github.com/ShenhanQian/GaussianAvatars

📺 Video: https://www.youtube.com/watch?v=lVEY78RwU_I

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #HeadAvatar #GaussianAvatars  #DynamicGaussians #3DModeling #AvatarGeneration #CVPR2024 #DeepLearning #Innovation

DmitryRyumin samusenps xenagarage wickedPixel clem victor pratikjadhav jonasmaltebecker

DmitryRyumin AngryBear9019 danielus Viczap


🏅 Quantized Embeddings are here! Unlike model quantization, embedding quantization is a post-processing step for embeddings that converts e.g. `float32` embeddings to binary or `int8` embeddings. This saves 32x or 4x memory & disk space, and these embeddings are much easier to compare!

Our results show 25-45x speedups in retrieval compared to full-size embeddings, while keeping 96% of the performance!

Learn more about it in our blogpost in collaboration with mixedbread.ai: https://huggingface.co/blog/embedding-quantization
Or try out our demo where we use quantized embeddings to let you search all of Wikipedia (yes, 41,000,000 texts) in 1 second on a CPU Space: https://huggingface.co/spaces/sentence-transformers/quantized-retrieval

jaggernaut007 snowdere osanseviero AtAndDev cnmoro mohammedbriman clem pascalnjue louisbrulenaudet iamrobotbear mrdbourke danielpark MichelleRuwen

samusenps osanseviero MarklarFromMarklar AtAndDev clem AiModelsMarket iamrobotbear EllenGu danielpark MichelleRuwen


🔥🆕🆕🔥 Dataset Drop: 4 KTO signal transformed versions of the highly loved Argilla DPO datasets.

KTO formats for:
- UltraFeedback Cleaned Binarized
- Distilabel Intel Orca
- Distilabel Capybara
- DPO mix

https://huggingface.co/collections/argilla/preference-datasets-for-kto-65f98314d7c1b04ab54d41a7

Paper claims :)

https://arxiv.org/abs/2402.01306

KTO matches or exceeds DPO performance at scales from 1B to 30B parameters.1 That is, taking a preference dataset of n DPO pairs and breaking it up into 2n examples for KTO can yield better generations, despite the model ostensibly learning from a weaker signal.

KTO can handle extreme data imbalances, matching DPO performance while using up to 90% fewer desirable examples (i.e., examples of good generations). Its success thus cannot be ascribed to the alignment data being sourced from a preference dataset.

When the pretrained model is sufficiently good, one can skip supervised finetuning and go straight to KTO without a loss in generation quality. In contrast, we find that without doing SFT first, DPO-aligned models are significantly worse at all scales.

Do you need something custom? Take a look at @davanstrien his guide on creating your own KTO dataset with Argilla and our community.

https://github.com/huggingface/data-is-better-together/tree/main/kto-preference

davanstrien ZennyKenny osanseviero giux78 clem


🤗 PEFT v0.10.0 release! 🔥🚀✨

Some highli📝ghts:
1. FSDP+QLoRA and DeepSpeed Stage-3+QLoRA
2. Layer expansion + LoRA
3. DoRA support for Conv2D layers and quantized bitsandbytes layers
4. New LoftQ utility
5. Batched inference for mixed LoRA adapters.

http://Answer.AI team in collaboration with bitsandbytes and Hugging Face 🤗 open sourced code enabling the usage of FSDP+QLoRA and explained the whole process in their insightful blogpost https://lnkd.in/g6jgfXyv. This is now integrated into Hugging Face ecosystem.

For an end-to-end example on FSDP+QLoRA, please refer https://lnkd.in/gT3yY-Rx.

For an end-to-end example on DeepSpeed Stage-3+QLoRA, please refer https://lnkd.in/gkt-xZRE.

With the PR https://lnkd.in/g5F348MN these changes are now upstreamed in https://lnkd.in/g5_MxYtY thanks to Wing Lian ! 🚀

Kudos to http://Answer.AI team,  Titus von Köller , Younes Belkada, Benjamin Bossan and Zachary Mueller for all the help without which this couldn't have been possible. 🤗

For efficient depthwise layer expansion akin to `passthrough` method of `mergekit` but without using additional memory and attaching LoRAs to it, refer to the details below! 🔥https://lnkd.in/ge95ztjA

Now DoRA is supported for Conv2D layers as well as bitsandbytes quantized layers ✨. For more details, please refer the below thread.
https://lnkd.in/gsJbuWPD

Now you can mix different LoRA adapters in a batch during inference which speeds-up the inference by avoiding computation of base model multiple times which would be the case for adaptive inference with batch_size=1! ⚡.
Details below. https://lnkd.in/gD-pcX_B

LoftQ reduces quantization error by appropriately initializing the LoRA adapter weights. Normally, this is a two-step process. Benjamin Bossan
added new util `replace_lora_weights_loftq` for LoftQ to use it on the fly with bnb.

For more details, refer to the release notes. 📝
https://lnkd.in/gg7-AmHA. As always, make sure losses go down and be happy to watch your model train!

osanseviero clefourrier DmitryRyumin nbroad samusenps adamo1139 giux78 louisbrulenaudet sergey21000

osanseviero damerajee masterfury SicariusSicariiStuff giux78


Look at the beauty in the video — four different embeddings on the same map! In another community blog post, I explore how you can use Nomic Atlas to view and clean your dataset. You can check it out here - https://huggingface.co/blog/visheratin/nomic-data-cleaning


Worked on a short blog post discussing how we semi-automated the release process of the `diffusers` library. The post delves deeper into the workflows responsible for:

* Publishing the package on Test PyPI and main PyPI servers.
* Notifying an internal Slack channel after a release is published on the repository.

Check it out here 👉
https://sayak.dev/posts/streamlined-releases.html


Evolutionary Optimization of Model Merging Recipes (2403.13187)

GraphWiz: An Instruction-Following Language Model for Graph Problems (2402.16029)

Diaries of Open Source. Part 7!

🔥Sakana releases Evolutionary Model Merge
Blog post: https://sakana.ai/evolutionary-model-merge/
Paper: https://huggingface.co/papers/2403.13187
Models and demo: https://hf.co/SakanaAI

🍞MixedBread releases new SoTA sentence embedding model
Announcement: https://www.mixedbread.ai/blog/mxbai-embed-large-v1
Model: https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1

🎥VideoMamba, a Mamba-based model for video understanding
Blog: https://hf.co/blog/vladbogo/video-mamba
Demo: https://huggingface.co/spaces/OpenGVLab/VideoMamba
Model: https://huggingface.co/OpenGVLab/VideoMamba

🔍 MathVerse, a visual math benchmark for multimodal LLMs
Paper page: https://mathverse-cuhk.github.io/
Dataset: https://huggingface.co/datasets/AI4Math/MathVerse
Paper: https://huggingface.co/papers/2403.14624

🧠GraphWiz, a family of instruct-tuned LLMs to solve graph problems
Repos: https://hf.co/GraphWiz
Paper: https://hf.co/papers/2402.16029

🪆NLLB-SigLIP-MRL: a combination of NLLB and SigLIP trained with Matryoshka representation learning
Model: https://hf.co/visheratin/nllb-siglip-mrl-large
Tweet: https://twitter.com/visheratin/status/1766643219909984734?s=46

🧍HDM and ProciGen: Template-free reconstruction of human-object interactions
Paper page: https://virtualhumans.mpi-inf.mpg.de/procigen-hdm/
Demo: https://hf.co/spaces/xiexh20/HDM-interaction-recon?logs=build
Models: https://hf.co/xiexh20/HDM-models

🌎Models and data around the world
EagleX 7B, multi-lingual RNN-based model https://hf.co/spaces/recursal/EagleX-7B-1.7T-Gradio-Demo
Tamil LLM https://hf.co/mervinpraison/tamil-large-language-model-7b-v1.0

osanseviero victor not-lain xiexh20 visheratin giux78 DmitryRyumin taufiqdp samusenps AtAndDev mmhamdy InferenceIllusionist clem

jucamohedano not-lain clefourrier xiexh20 visheratin giux78 AtAndDev mmhamdy dillfrescott InferenceIllusionist clem


How to let LLM acquire the Agentic Capability?

Previous answers are direct imitation learning by collecting agentic data such as tool calling history (inefficient and introduces format hallucination).
Agent-FLAN tells a different view:
- Eliciting the foundational capability (e.g., reasoning, retrieval, and instruction following) is more important
- Using chat data is more effective with less side effects than tool calling history

Dataset: https://huggingface.co/datasets/internlm/Agent-FLAN
HF Model: https://huggingface.co/internlm/Agent-FLAN-7b
Paper: https://huggingface.co/papers/2403.12881
Project page：https://internlm.github.io/Agent-FLAN/


Here it is, an Adaptive AI Assistant (AAA) only made possible through open-source, incredibly inspiring, well done to the Open Interpreter team👏🏼👏🏼

Glad this one did not start with big tech :)

Code:  https://github.com/OpenInterpreter/01

EidosL awacke1 alielfilali01 Seuriin clem elsiddik


LLaVA-NeXT is recently merged to Hugging Face transformers and it outperforms many of the closed source models like Gemini on various benchmarks 🤩  Let's take a look!
Demo: https://huggingface.co/spaces/merve/llava-next
Notebook: https://colab.research.google.com/drive/1afNudu72SNWZCYtCVrRlb9T9Vj9CFJEK?usp=sharing
LLaVA is essentially a vision-language model that consists of ViT-based CLIP encoder, a MLP projection and Vicuna as decoder ✨
LLaVA 1.5 was released with Vicuna, but LLaVA NeXT (1.6) is released with four different LLMs:
- Nous-Hermes-Yi-34B
- Mistral-7B
- Vicuna 7B & 13B
Mistral and Nous-Hermes-Yi-34B are performing better and have better commercial use.
Moreover, according to authors' findings, the improvements comes from more diverse and high quality data mixture and dynamic high resolution.
LLaVA based on Nous-Hermes-Yi-34B outperforms many other models, including Gemini in various multimodal understanding and generation benchmarks 😊

osanseviero samusenps yesdeepakmittalTA Dlbk InferenceIllusionist nqtruong mirellyssl acamilogg88 clem LucienL shing3232 jarvisx17


New NVIDIA GTC24 paper 🎊

We generate high-quality 3D assets in only 400ms from text by combining (a) amortized optimization for speed, (b) surface rendering for quality, and (c) 3D data for robustness.

☕ LATTE3D project details: https://research.nvidia.com/labs/toronto-ai/LATTE3D/

ZennyKenny osanseviero merve NickyNicky Dlbk bvk1ng lorraine2


🌊LaVague can compile Action Plans into actionable code to browse the internet!

In this example, you can see how an action plan with natural language instructions can be “compiled” into executable Selenium code!

🤖This shows the potential of #LAM (Large Action Models) to perform actions for us and automate mechanical tasks.
This example leverages a local embedding model and OpenAI GPT-3.5, but we support many options, including local ones with Gemma!
You can try this in our docs: https://docs.lavague.ai/en/latest/

LaVague is an open-source Large Action Model framework to automate automation. If you are interested in helping us on our mission to democratize automation tooling for devs, don’t hesitate to visit our GitHub (https://github.com/lavague-ai/LaVague) or Discord (https://discord.gg/SDxn9KpqX9)!

AjasLearner radames osanseviero merve samusenps kramp clefourrier louisbrulenaudet TechxGenus mathiasn1 clem


Mora: Enabling Generalist Video Generation via A Multi-Agent Framework (2403.13248)

Mora

Enabling Generalist Video Generation via A Multi-Agent Framework

https://huggingface.co/papers/2403.13248

Sora is the first large-scale generalist video generation model that garnered significant attention across society. Since its launch by OpenAI in February 2024, no other video generation models have paralleled {Sora}'s performance or its capacity to support a broad spectrum of video generation tasks. Additionally, there are only a few fully published video generation models, with the majority being closed-source. To address this gap, this paper proposes a new multi-agent framework Mora, which incorporates several advanced visual AI agents to replicate generalist video generation demonstrated by Sora. In particular, Mora can utilize multiple visual agents and successfully mimic Sora's video generation capabilities in various tasks, such as (1) text-to-video generation, (2) text-conditional image-to-video generation, (3) extend generated videos, (4) video-to-video editing, (5) connect videos and (6) simulate digital worlds. Our extensive experimental results show that Mora achieves performance that is proximate to that of Sora in various tasks. However, there exists an obvious performance gap between our work and Sora when assessed holistically. In summary, we hope this project can guide the future trajectory of video generation through collaborative AI agents.

osanseviero notune merve dn6 AtAndDev ahmed-ai Shinku Lewdiculous uglysonic3121 clem alielfilali01

merve samusenps AtAndDev AIIAR ahmed-ai halilozturkci clem


How talkative is your chatbot about your internal data? 😬

As more chatbots get deployed in production, with access to internal databases, we need to make sure they don't leak private information to anyone interacting with them.

The Lighthouz AI team therefore introduced the Chatbot Guardrails Arena to stress test models and see how well guarded your private information is.
Anyone can try to make models reveal information they should not share 😈
(which is quite fun to do for the strongest models)!

The votes will then be gathered to create an Elo ranking of the safest models with respect to PII.

In the future, with the support of the community, this arena could inform safety choices that company make, when choosing models and guardrails on their resistance to adversarial attacks.
It's also a good way to easily demonstrate the limitations of current systems!

Check out the arena: https://huggingface.co/spaces/lighthouzai/guardrails-arena
Learn more in the blog: https://huggingface.co/blog/arena-lighthouz


Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic
  Gaussians (2312.03029)

🚀🎭🌟 New Research Alert - CVPR 2024! 🌟 🎭🚀
📄 Title: Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians 🔝

📝 Description: Gaussian Head Avatar is a method for generating highly detailed 3D head avatars using dynamic Gaussian functions controlled by a neural network, ensuring ultra-high quality visualization even under limited viewpoints.

👥 Authors: Yuelang Xu, @ben55, Zhe Li, @HongwenZhang, @wanglz14, Zerong Zheng, and @YebinLiu

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2312.03029

🌐 Github Page: https://yuelangx.github.io/gaussianheadavatar
📁 Repository: https://github.com/YuelangX/Gaussian-Head-Avatar

📺 Video: https://www.youtube.com/watch?v=kvrrI3EoM5g

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #HeadAvatar #DynamicGaussians #3DModeling #AvatarGeneration #CVPR2024 #DeepLearning #Innovation

DmitryRyumin mrfakename osanseviero samusenps akhaliq eieihihi Katiyar48

DmitryRyumin osanseviero akhaliq clefourrier


Welcome to LLMLingua-2, a small-size yet powerful prompt compression method trained via data distillation from GPT-4 for token classification with a BERT-level encoder, excels in task-agnostic compression. It surpasses LLMLingua in handling out-of-domain data, offering 3x-6x faster performance. @qianhuiwu

website: https://llmlingua.com/llmlingua2.html
code: https://github.com/microsoft/LLMLingua
demo: https://huggingface.co/spaces/microsoft/llmlingua-2

victor osanseviero alexandreteles acamilogg88 taisazero Qwoook hieutrungdao


I just uploaded a Dataset that adds AI descriptions to top trending Spaces! It is really useful to find awesome Spaces 🚀

https://huggingface.co/datasets/victor/hf-spaces-with-descriptions/viewer/default/train

I used HF Serverless Endpoints (with Mixtral) to get the descriptions, it was seamless to do! btw check out the new Serverless Endpoints dashboard - it's awesome!

osanseviero radames samusenps m-ric nmnijilkhan pierrci


Just released a dataset with 1.5M image question/answers! https://huggingface.co/datasets/vikhyatk/lnqa

Felladrin btbytes femboysLover AIIAR ajibawa-2023 samusenps radames clefourrier kristaller486 VictorSanh smangrul zelus82 Sylvestre


One-Step Image Translation with Text-to-Image Models (2403.12036)

Testing new pix2pix-Turbo in real-time, very interesting GAN architecture that leverages SD-Turbo model. Here I'm using edge2image LoRA single-step inference 🤯

It's very interesting how ControlNet Canny quality is comparable, but in a single step. Looking forward to when they release the code: https://github.com/GaParmar/img2img-turbo/issues/1

I've been keeping a list of fast diffusion model pipelines together with this real-time websocket app. Have a look if you want to test it locally, or check out the demo here on Spaces.

https://huggingface.co/spaces/radames/real-time-pix2pix-turbo

Github app:
https://github.com/radames/Real-Time-Latent-Consistency-Model/

You can also check the authors img2img sketch model here

https://huggingface.co/spaces/gparmar/img2img-turbo-sketch

Refs:
https://huggingface.co/papers/2403.12036

cc @gparmar @junyanz

samusenps xsa-dev osanseviero VeyVey abidlabs ChangWWWWq JoeyKo1004 digitalcmartinez gparmar Csplk sbarman25 gym890


We've just published a detailed blog post on the creation of Cosmopedia dataset. We hope this will provide insights about generating synthetic data at scale for pre-training.
https://huggingface.co/blog/cosmopedia

Here are some key takeaways:
🎯 Prompt curation is crucial: we want to cover many topics with few duplicates.
📚 You can leverage various resources for diversity: using different seed data, generation formats, and target audiences.
⚙ The importance of a good technical stack: for scalable generations with tools like llm-swarm and fast model training and evaluation.

Have a good read!

alielfilali01 anton-l vwxyzjn kavalciemre samusenps osanseviero lvwerra xu3kev avinash02 thomwolf fpaupier manu muhtasham mjoshi malmenea AdinaY victor clefourrier molonelaveh

alielfilali01 anton-l vwxyzjn samusenps osanseviero xu3kev mmhamdy thomwolf muhtasham aloobun Salwa-Zeitoun AdinaY Dflare molonelaveh timpearce svallory

alielfilali01 anton-l vwxyzjn severo osanseviero xu3kev mmhamdy thomwolf muhtasham AdinaY ajibawa-2023 InferenceIllusionist mlabonne on1onmangoes molonelaveh


The Era of 1-bit LLMs: Training Tips, Code and FAQ

https://github.com/microsoft/unilm/blob/master/bitnet/The-Era-of-1-bit-LLMs__Training_Tips_Code_FAQ.pdf

We present details and tips for training 1-bit LLMs. We also provide additional experiments and results that were not reported and responses to questions regarding the "The-Era-of-1-bit-LLM" paper. Finally, we include the official PyTorch implementation of BitNet (b1.58 and b1) for future research and development of 1-bit LLMs.

HDiffusion kaykyramos osanseviero ndavidson samusenps xu3kev avinash02 NePe victor bunnycore impactframes AtAndDev alielfilali01 Heihai

thegenerality xu3kev Smith42 AtAndDev damerajee InferenceIllusionist Heihai


Announcing today the release of Common Corpus, the largest collection of fully open corpus on HuggingFace: nearly 500b words (600-700b tokens) in public domain.

https://huggingface.co/collections/PleIAs/common-corpus-65d46e3ea3980fdcd66a5613

Common corpus is an international initiative coordinated by @pleias_fr with the support of the state start-up LANGU:IA (start-up d’Etat), supported by the French Ministry of Culture and DINUM and the involvement of the open science LLM community (Occiglot, Eleuther AI) and cultural heritage researchers.

We aim to create the same kind of ecosystem there is now for fine tuning at the pretraining stage, by creating a strong commons without copyright issues or "trade secret" gatekeeping. Contrary to what many AI companies say, Common Corpus shows it is possible to train Large Language Models on fully open corpus. Due to the complexity of copyright check, we have only released a partial amount of the text we hold and will release way more in the months.

Common Corpus is multilingual. It also includes to date the largest open collections in French (110 billion words), German (30 billion words), Spanish (23 billion words), Dutch (18 billion words), Italian (10 billion words) as well as a very long tail of middle to low resource languages.

Our conviction is that open corpora make future models more inclusive, democratic, and respectful of cultural diversity, as well as more qualitative. Common Corpus holds many long texts in book form, editorialized, with reasoning rich content that have never been used to date for LLM pretraining.

Common Corpus is an ongoing work and still need to get enhanced and completed. Sharing is caring: Common Corpus still needs more care to become "a common" like Wikipedia or Wikisource.

https://huggingface.co/blog/Pclanglais/common-corpus

julien-c storytracer victor osanseviero giux78 Felladrin carbonbasedLLM Lewdiculous SerialKicked codito pedevineau J-Hansen metasj