OpenCodeInterpreter: Integrating Code Generation with Execution and
  Refinement (2402.14658)

Diaries of Open Source. Part 3! OS goes to the moon!

💻  OpenCodeInterpreter, a family of very powerful code generation models
Models: https://huggingface.co/collections/m-a-p/opencodeinterpreter-65d312f6f88da990a64da456
Paper: https://huggingface.co/papers/2402.14658
Demo https://huggingface.co/spaces/m-a-p/OpenCodeInterpreter_demo

🔷🔶Zephyr 7B Gemma, Gemma fine-tuned with the Zephyr recipe
Model: https://huggingface.co/HuggingFaceH4/zephyr-7b-gemma-v0.1
Demo: https://huggingface.co/spaces/HuggingFaceH4/zephyr-7b-gemma-chat
GH Repo: https://github.com/huggingface/alignment-handbook

🪆The MixedBread folks released a 2D Matryoshka text embedding model, which means you can dynamically change the embedding size and layer counts
Model: https://huggingface.co/mixedbread-ai/mxbai-embed-2d-large-v1
Release blog post: https://www.mixedbread.ai/blog/mxbai-embed-2d-large-v1

🐋Microsoft released Orca Math, which includes 200K grade school math problems
Dataset: https://huggingface.co/datasets/microsoft/orca-math-word-problems-200k

🥷IBM silently released Merlinite, a cool model trained on Mixtral-generated synthetic data using a novel LAB method https://huggingface.co/ibm/merlinite-7b

🌚 Moondream2 - a small vision language model to run on-device!
Model: https://huggingface.co/vikhyatk/moondream2
Demo: https://huggingface.co/spaces/vikhyatk/moondream2

🏙CityDreamer: 3D City Generation
Demo: https://huggingface.co/spaces/hzxie/city-dreamer
Repo: https://github.com/hzxie/city-dreamer
Model: https://huggingface.co/hzxie/city-dreamer

🌏ML in all languages
Sailor, a family of South-East Asian languages models https://huggingface.co/collections/sail/sailor-language-models-65e19a749f978976f1959825
Samvaad dataset, which includes 140k QA pairs in Hindi, Bengali, Marathi, Tamil, Telugu, Oriya, Punjabi, and Gujarati https://huggingface.co/datasets/GenVRadmin/Samvaad-Mixed-Language-2

You can see the previous part at https://huggingface.co/posts/osanseviero/674644082063278

DmitryRyumin fffiloni samusenps vladbogo unclecode visheratin ajibawa-2023 mvaloatto mikonvergence Makya


I have just published my first blog post.

While FlashAttention has been readily integrated into HuggingFace transformers, there are much higher gains to be had (at least theoretically) for finetuning models with examples of variable sequence lengths in a batch.

For a deeper dive, please go through my blog at https://huggingface.co/blog/mayank-mishra/padding-free-transformer.

Kukedlc osanseviero tomaarsen samusenps eramax mayank-mishra julien-c

damerajee julien-c mayank-mishra jeffboudier


GPAvatar: Generalizable and Precise Head Avatar from Image(s) (2401.10215)

🚀🎭🌟 New Research Alert - ICLR 2024! 🌟🎭 🚀
📄 Title: GPAvatar: Generalizable and Precise Head Avatar from Image(s) 🌟🚀

📝 Description: GPAvatar's objective is to faithfully replicate head avatars while providing precise control over expressions and postures.

👥 Authors: Xuangeng Chu et al.

📅 Conference: ICLR, May 7-11, 2024 | Vienna, Austria 🇦🇹

🔗 Paper: https://huggingface.co/papers/2401.10215

🔗 Github Page: https://xg-chu.github.io/project_gpavatar
🔗 Repository: https://github.com/xg-chu/GPAvatar

🔗 Video: https://www.youtube.com/watch?v=7A3DMaB6Zk0

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #GPAvatar #MTA #Synthesis #LipSyncing #Expressions #HighResolutionVideos #ICLR2024 #DeepLearning #Animation #Innovation

DmitryRyumin osanseviero samusenps philipp-zettl rreed-pha


Hi!
We release a series of LLMs named GraphWiz https://huggingface.co/GraphWiz/, which is designed for solving different graph problems and outputs their CoT. GraphWiz has surpassed GPT-4 by a large margin in 9 tasks.

Let us have a try!

Nuo97 osanseviero Kukedlc samusenps kramp Androidkit yoeldcd


Last day on Spaces of the Week ,
and we made it to last place on trending.
i really thought it couldnt get any better, but i'm crying ! 😭

The thing i like the most about ZeroGPU , `import spaces` , is that i dont have to always check to see if someone decided to test if i have hard character limits , and it reloads the application flawlessly .

drop a like on my spaces here :
Spaces of the Week : https://huggingface.co/spaces/tonic/starcoder2
9 other ZeroGPU demos :  https://huggingface.co/tonic


I remember very well that about two years ago, 0-shot named entity recognition (i.e. where you can choose any labels on the fly) was completely infeasible. Fast forward a year, and https://huggingface.co/Universal-NER/UniNER-7B-all surprised me by showing that 0-shot NER is possible! However, I had a bunch of concerns that prevented me from ever adopting it myself. For example, the model was 7B parameters, only worked with 1 custom label at a time, and it had a cc-by-nc-4.0 license.

Since then, a little known research paper introduced GLiNER, which was a modified & finetuned variant of the https://huggingface.co/microsoft/deberta-v3-base line of models. Notably, GLiNER outperforms UniNER-7B, despite being almost 2 orders of magnitude smaller! It also allows for multiple labels at once, supports nested NER, and the models are Apache 2.0.

Very recently, the models were uploaded to Hugging Face, and I was inspired to create a demo for the English model. The demo runs on CPU, and can still very efficiently compute labels with great performance. I'm very impressed at the models.

There are two models right now:
* base (english): https://huggingface.co/urchade/gliner_base
* multi (multilingual): https://huggingface.co/urchade/gliner_multi

And my demo to experiment with the base model can be found here: https://huggingface.co/spaces/tomaarsen/gliner_base

osanseviero MexIvanov urchade rajistics samusenps ajibawa-2023 danielus nickprock polodealvarado deepsh2207 luqman2511 chokamp Poulain Godspeed-AI eek louisguitton


Multi-LoRA Composition for Image Generation (2402.16843)

"Multi-LoRA Composition for Image Generation" introduces two new approaches for combining multiple visual elements in text-to-image generation using Low-Rank Adaptations (LoRAs)! 🎨

Key Points:
* Proposes two methods - LoRA Switch and LoRA Composite - that activate/combine LoRAs during the denoising process rather than merging weights
* LoRA Switch cycles through different LoRAs at each step, while LoRA Composite averages guidance from all LoRAs simultaneously

Paper: https://huggingface.co/papers/2402.16843
Project page: https://maszhongming.github.io/Multi-LoRA-Composition

Congrats to the authors for their work!

DmitryRyumin osanseviero samusenps mvaloatto femboysLover


Introducing https://huggingface.co/datasets/davanstrien/cosmopedia_chat (v0.0.1), my first experiment using the new NousResearch Genstruct model https://huggingface.co/NousResearch/Genstruct-7B

This dataset uses a subset of https://huggingface.co/datasets/HuggingFaceTB/cosmopedia, a synthetic textbook-quality dataset, and Genstruct to generate user/assistant response pairs.

My current results are mixed, but I'm excited to see how much work is happening around synthetic data generation in the community.  Most crucial next step is working more on data filtering from cosmopedia.

Massive thanks to @euclaise @teknium and the other NouseResearch folks for sharing this model ❤


Learning to Generate Instruction Tuning Datasets for Zero-Shot Task
  Adaptation (2402.18334)

Create synthetic instruction datasets using open source LLM's and bonito🐟!

With Bonito, you can generate synthetic datasets for a wide variety of supported tasks.

The Bonito model introduces a novel approach for conditional task generation, transforming unannotated text into task-specific training datasets to facilitate zero-shot adaptation of large language models on specialized data.

This methodology not only improves the adaptability of LLMs to new domains but also showcases the effectiveness of synthetic instruction tuning datasets in achieving substantial performance gains.

AutoBonito🐟: https://colab.research.google.com/drive/1l9zh_VX0X4ylbzpGckCjH5yEflFsLW04?usp=sharing
Original Repo: https://github.com/BatsResearch/bonito?tab=readme-ov-file
Paper: https://huggingface.co/papers/2402.18334

samusenps pabloce clem nihalnayak damerajee jayomb nanyy1025


PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K
  Text-to-Image Generation (2403.04692)

PixArt-Σ

Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation

https://huggingface.co/papers/2403.04692

In this paper, we introduce PixArt-\Sigma, a Diffusion Transformer model~(DiT) capable of directly generating images at 4K resolution. PixArt-\Sigma represents a significant advancement over its predecessor, PixArt-\alpha, offering images of markedly higher fidelity and improved alignment with text prompts. A key feature of PixArt-\Sigma is its training efficiency. Leveraging the foundational pre-training of PixArt-\alpha, it evolves from the `weaker' baseline to a `stronger' model via incorporating higher quality data, a process we term "weak-to-strong training". The advancements in PixArt-\Sigma are twofold: (1) High-Quality Training Data: PixArt-\Sigma incorporates superior-quality image data, paired with more precise and detailed image captions. (2) Efficient Token Compression: we propose a novel attention module within the DiT framework that compresses both keys and values, significantly improving efficiency and facilitating ultra-high-resolution image generation. Thanks to these improvements, PixArt-\Sigma achieves superior image quality and user prompt adherence capabilities with significantly smaller model size (0.6B parameters) than existing text-to-image diffusion models, such as SDXL (2.6B parameters) and SD Cascade (5.1B parameters). Moreover, PixArt-\Sigma's capability to generate 4K images supports the creation of high-resolution posters and wallpapers, efficiently bolstering the production of high-quality visual content in industries such as film and gaming.

samusenps clem mvaloatto victor thomwolf ianyeung DeathGodlike


As promised, a video on the new Major TOM dataset on HF: https://youtu.be/KonWxQ1mCpA?si=tp3Cz6-F0exfgdx3

If you prefer audio, there is a podcast too: https://www.satellite-image-deep-learning.com/p/major-tom-expandable-eo-datasets

I think this is just the start for remote sensing datasets on HuggingFace..!


🗄 Massive data release on the HF Hub for 75 languages!

https://huggingface.co/datasets/BramVanroy/hplt_monolingual_v1_2

In December of last year, HPLT (https://hplt-project.org/) released version 1.2 of their dataset. It covers web-crawled data of 75 languages!, in the raw format as well as deduplicated and cleaned sections. In total, we're talking about over 40TB of data! This data was already accessible via their website but I figured the accessibility could be improved by an integration with Hugging Face tooling. 🤗 So I added the dataset here to the Hugging Face hub, enabing direct use in your conventional training pipelines for LLMs or other language technologies. The data will automatically be downloaded and optimised with just one line of code:

```python
load_dataset("BramVanroy/hplt_mono_v1_2", "nl_cleaned")
```
Let's use this big blob of data to build something awesome in our languages! 🥳

stefan-it samusenps ajibawa-2023 clem osanseviero ymoslem damerajee rafaelpierrehf


We just released bitsandbytes==0.43.0 📦 , with these significant new additions:

‣ 🛫 FSDP+QLoRA support (alpha release)
  ◦ now anyone with 2 powerful gaming GPUs can fine-tune 70B param models at home!
  ◦ in collab with Jeremy Howard + team @ answer.ai
  ◦ answer.ai blogpost: https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html
  ◦ example repo: https://github.com/AnswerDotAI/fsdp_qlora/

‣ 🌈⊞ Official Windows support
  ◦ now via simple `pip install bitsandbytes>=0.43.0`

‣ 📄 Huge docs update:
  ◦ https://huggingface.co/docs/bitsandbytes/main
  ◦ Be sure to check out the optimizers and the API docs
  ◦ ... even more upcoming ...

Under the hood there we have many other improvements, due to extensive maintenance activity, community contributions by super active + knowledgable volunteers ✨ 🚀 and the official sponsorship by Hugging Face that makes all this possible 🤗 ❤ 🌍

We would greatly appreciate any further community contributions, be it to help with refactorings, exterminating flaky tests, writing doc-strings, tutorials, new features. Don't be shy, just contact us and we see where this leads us:
https://github.com/TimDettmers/bitsandbytes/discussions

Have a great weekend everyone!

Titus-von-Koeller mdouglas clem osanseviero rganti ajibawa-2023 julien-c t1u1 ybelkada cstr tolgacangoz hiyouga danielus smangrul

Djruffkutz clem osanseviero samusenps julien-c Theli ybelkada hiyouga smangrul


Speaking of the missing piece in today’s generative AI: Reasoning (or more appropriately, the proper use of Common-Sense)

Human Intelligence is hinged on the brain’s ability to learn vast amounts of background knowledge about the world just by passively observing it. Such common-sense information is believed to be the enabler of intelligent behavior (planning, reasoning and grounding).

Unusual question: how do we actually learn common-sense knowledge?

Unusual opinion: I personally believe we haven’t fully understand how the brain learns, thus cannot get machines to mimic how we learn.

Well so far AI godfather Prof. Yann have quite a promising vision of how machines can learn world models like we humans do. Excited to share his vision after giving I-JEPA a read.

I-JEPA (Image-based Joint-Embedding Predictive Architecture) is a novel approach to self-supervised learning from images. This method focuses on learning semantic image features without relying on pre-set rules based on manual data changes. Instead, I-JEPA predicts the representations of multiple target blocks within a single image using a single context block.

The I-JEPA architecture consists of a context encoder, a target encoder, and a predictor. The context encoder extracts context features from a context block, while the target encoder extracts target features from the target blocks. The predictor then uses the context features to predict the target features.

One of the main advantages of I-JEPA is that it is non-generative, meaning it does not rely on pre-set rules based on manual data changes. It also uses multi-block masking, which allows it to learn semantic representations more effectively.

This is very promising, hopefully we can look back this one day and amuse at how we got it right.

Paper: https://arxiv.org/abs/2301.08243
Code: https://github.com/facebookresearch/ijepa


People in Paris 🇫🇷 🥐

Next week we'll be hosting our first Post-Deployment Data Science Meetup in Paris!

My boss will be talking about Quantifying the Impact of Data Drift on Model
Performance. 👀

The event is completely free, and there's only space for 50 people, so if you are interested, RSVP as soon as possible 🤗

🗓 Thursday, March 14
🕠 5:30 PM - 8:30 PM GMT+1
🔗 RSVP: https://lu.ma/postdeploymentparis


Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis (2401.08503)

🚀🎭🌟 New Research Alert - ICLR 2024! 🌟🎭 🚀
📄 Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis 🌟🚀

👥 Authors: Zhenhui Ye et al.

📅 Conference: ICLR, May 7-11, 2024 | Vienna, Austria 🇦🇹

🔗 Paper: https://huggingface.co/papers/2401.08503

🔗 Github Page: https://real3dportrait.github.io/
🔗 Repository: https://github.com/yerfor/Real3DPortrait

🔥 Model 🤖: https://huggingface.co/ameerazam08/Real3DPortrait

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #Real3D-Potrait #I2P #HTB-SR #A2M #Synthesis #LipSyncing #HighResolutionVideos #ICLR2024 #DeepLearning #Animation #Innovation

DmitryRyumin samusenps clem osanseviero MiSTe-R Saba99

samusenps clem osanseviero DmitryRyumin MiSTe-R


 The past year has seen a rapid pace of delivery of open-to-use Large Language Models (LLMs) for Generative AI. Simultaneously, a number of service providers have bootstrapped, which provide cloud based APIs for generative inference using these LLMs. The list is long: Together.AI, Fireworks.AI, Mosaic ML (Databricks), Anyscale , Perplexity Labs... Assuming that you are fine with privacy and security issues that sending your data to a cloud hosted solution bring, these providers can help launch your Generative AI use case with little friction.

Given that all the cloud based providers provide access to the same/similar open models, how do you distinguish between them? One way is to look at the performance vs cost tradeoff. This post talks about performance.

To understand performance, one needs benchmarks and metrics. There exist a number of benchmarks and leader boards to compare solutions. But, before you start comparing data points flying across the internet, I suggest you to consider the following.

1) Play with your own benchmark: You need to understand the performance for your use case, not some random workload a off-the-shelf benchmark seeks to represent and its authors dreamed off. You need to build and run your own benchmark. You could either understand and re-work the publicly available benchmarks like LLMPerf, or design your own one. Contrary to wisdom, writing a benchmark grounds up isn't hard. I wrote a simple benchmark called GenAI-Bench to compare the response time of cloud providers. You can do it too.
2) Understand the metrics you care about. Is your use case a chatbot which needs a quick turnaround for the first word? Or do you need to parse a large number of documents and get a summary over-night. Correctly identifying what you care for will help you narrow down the best provider for your needs.
3) Keep generating data points periodically. Cloud providers routinely update their software. Metrics change. Measure them periodically.


After the Supervised Fine-Tuning (SFT) phase, we observed a notable degradation in the instruction-following capabilities of the LLaVA Multi-Modal Large Language Model (MM-LLM). To address this issue, we introduced a 6K-entry VQA preference dataset and employed Direct Preference Optimization (DPO), alongside testing other algorithms such as Rejection Sampling and SteerLM, to enhance instruction-following proficiency. Our methodology not only fully restored the language following capabilities of LLaVa on the MT-Bench but also outperformed LLaVA-RLHF and Vicuna. Additionally, our approach extended to visual VQA tasks, as demonstrated by significant performance improvements on MM-Vet and LLaVa-Bench. An interesting observation was that, compared to models using distilled SFT, our method showed substantial out-of-distribution improvements.

https://arxiv.org/abs/2402.10884
Model available
https://huggingface.co/alexshengzhili/llava-v1.5-13b-dpo
GitHub:
https://github.com/findalexli/mllm-dpo/edit/main/README.MD


Dear music lovers 🕺,

MusicLang Space is now live: https://huggingface.co/spaces/musiclang/README

MusicLang is a controllable model for music generation:

> 🦙 Discover the LLAMA2 architecture, trained from scratch for symbolic music generation, ensuring exceptional quality;
> 👨🎨 Unleash your creativity by extending an existing music, or create new ones from scratch;
> 🤖 Integrate MusicLang into your applications, with an inference optimized for CPUs written in C, other integrations and optimizations coming soon.

In the space, you’ll find :

1⃣   MusicLang foundation model: our fondation model for creating and generating original midi soundtracks https://huggingface.co/musiclang/musiclang-v2;

2⃣  MusicLang predict: our AI prediction api of the MusicLang package https://github.com/musiclang/musiclang_predict?tab=readme-ov-file;

3⃣  MusicLang Language:a new language for tonal music. This language allows composers to load, write, transform and predict symbolic music in a simple, condensed and high level manner  https://github.com/MusicLang/musiclang;

4⃣ MusicLang Demo Space: https://huggingface.co/spaces/musiclang/musiclang-predict

5⃣ Our Colab: https://colab.research.google.com/drive/1MA2mek826c05BjbWk2nRkVv2rW7kIU_S?usp=sharing

Help us share the future of music composition! Spread the word, show your support by adding a star or contribute to our project. ⭐✨

Music Sounds Definitely Better with You 🎶 🖤

cc @floriangardin @MehdiLeZ @reach-vb

Thanks a lot,

The MusicLang team ❤

floriangardin reach-vb samusenps Bils kramp fffiloni clem osanseviero victor MehdiLeZ asigalov61 RadMann


GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection (2403.03507)

GaLore

Memory-Efficient LLM Training by Gradient Low-Rank Projection

https://huggingface.co/papers/2403.03507

Training Large Language Models (LLMs) presents significant memory challenges, predominantly due to the growing size of weights and optimizer states. Common memory-reduction approaches, such as low-rank adaptation (LoRA), add a trainable low-rank matrix to the frozen pre-trained weight in each layer, reducing trainable parameters and optimizer states. However, such approaches typically underperform training with full-rank weights in both pre-training and fine-tuning stages since they limit the parameter search to a low-rank subspace and alter the training dynamics, and further, may require full-rank warm start. In this work, we propose Gradient Low-Rank Projection (GaLore), a training strategy that allows full-parameter learning but is more memory-efficient than common low-rank adaptation methods such as LoRA. Our approach reduces memory usage by up to 65.5% in optimizer states while maintaining both efficiency and performance for pre-training on LLaMA 1B and 7B architectures with C4 dataset with up to 19.7B tokens, and on fine-tuning RoBERTa on GLUE tasks. Our 8-bit GaLore further reduces optimizer memory by up to 82.5% and total training memory by 63.3%, compared to a BF16 baseline. Notably, we demonstrate, for the first time, the feasibility of pre-training a 7B model on consumer GPUs with 24GB memory (e.g., NVIDIA RTX 4090) without model parallel, checkpointing, or offloading strategies.

osanseviero notsahil andysalerno nofreewill femboysLover OneLoneTurnip boapps arcdyn megataro bytesizedllm clem jthk-HF rganti shing3232 mrm8488


BioT5: Enriching Cross-modal Integration in Biology with Chemical
  Knowledge and Natural Language Associations (2310.07276)

BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations

BioT5 achieves superior performance on various biological tasks by integrating natural language. See more details at:

Paper: https://huggingface.co/papers/2310.07276
Code: https://github.com/QizhiPei/BioT5
Model (base): https://huggingface.co/QizhiPei/biot5-base
Model (molecule captioning): https://huggingface.co/QizhiPei/biot5-base-mol2text
Model (Text-based Molecule Design): https://huggingface.co/QizhiPei/biot5-base-text2mol

osanseviero victor katielink samusenps Tonic QizhiPei clem


Extreme Compression of Large Language Models via Additive Quantization (2401.06118)

QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models (2310.16795)

Check out quantized weights from ISTA-DAS Lab directly in their organisation page: https://huggingface.co/ISTA-DASLab ! With official weights of AQLM (for 2bit quantization) & QMoE (1-bit MoE quantization)

Read more about these techniques below:

AQLM paper: https://hf.co/papers/2401.06118
QMoE: https://huggingface.co/papers/2310.16795

Some useful links below:

AQLM repo: https://github.com/Vahe1994/AQLM
How to use AQLM & transformers: https://huggingface.co/docs/transformers/quantization#aqlm
How to use AQLM & PEFT: https://huggingface.co/docs/peft/developer_guides/quantization#aqlm-quantizaion

Great work from @BlackSamorez and team !

alielfilali01 marcsun13 osanseviero samusenps FM-1976 mayank-mishra clem goodgame dillfrescott


🚀 Major Update: OpenLLM Turkish Benchmarks & Leaderboard Launch! 🚀

Exciting news for the Hugging Face community! I'm thrilled to announce the launch of my fully translated OpenLLM Benchmarks in Turkish, accompanied by my innovative leaderboard, ready to highlight the capabilities of Turkish language models. This marks a landmark achievement in supporting and advancing Turkish AI research.

What’s New:

📚 Complete OpenLLM Benchmarks in Turkish: Dive into my comprehensive suite of benchmarks, now available for thorough evaluation of Turkish LLMs.

📈 Live Leaderboard: Explore my live leaderboard showcasing the progress and excellence in Turkish language AI. (Note: Current evaluations are conducted manually but are consistently updated.)

Partnership Invitation:

🤝 Join My Automation Mission: I'm on the lookout for partners to help transition from manual to automated leaderboard evaluations. Your support can catalyze real-time, streamlined assessments, pushing Turkish LLMs to new heights.
Key Resources:

📚 Explore the Turkish OpenLLM Collection: (https://huggingface.co/collections/malhajar/openllmturkishleadboard-datasets-65e5854490a87c0f2670ec18)

🏆 Discover the Leaderboard: (https://huggingface.co/spaces/malhajar/OpenLLMTurkishLeaderboard)

Get Involved:

💡 Share Your Models: Contribute to the burgeoning field of Turkish AI, showcasing your work and contributing to the collective progress.

Let's unite to propel Turkish AI forward and set a precedent for the global community. Stay tuned as I plan to expand these efforts to other languages, further enriching the AI ecosystem!

Join this groundbreaking endeavor and let’s shape the future of AI together! 🌐

#TurkishLLM #AI #MachineLearning #LanguageModels #OpenLLM #HuggingFace

badayvedat osanseviero clefourrier Wauplin uisikdag samusenps comarproject thomwolf clem Yhyu13 ahmet27 mohammedbriman

kyo-takano victor Wauplin comarproject thomwolf HamzaMuaz ahmet27


`mamba` is now available in transformers.  Thanks to @tridao and @albertgu  for this brilliant model! 🚀 and the amazing `mamba-ssm` kernels powering this!
Checkout the collection here:
https://huggingface.co/collections/state-spaces/transformers-compatible-mamba-65e7b40ab87e5297e45ae406

lysandre osanseviero susnato victor ZetangForward samusenps chansung tomaarsen ceyda zjr2000


Anchor Large Language Models: Up to 99% KV cache reduction!

paper: https://arxiv.org/pdf/2402.07616.pdf

osanseviero damerajee samusenps pangjh3 clem Excursions Tonic


LLM “Patchnization”

Minimal, clean code for video/image "patchnization" - a process commonly used in tokenizing visual data for use in a Transformer encoder.

Code: https://github.com/Jaykef/min-patchnizer

The code above, first extracts still images (frames) from a video, splits the image frames into smaller fixed-size patches, linearly embeds each of them, adds position embeddings and then saves the resulting sequence of vectors for use in a Vision Transformer encoder. I tried training the resulting sequence vectors with Karpathy's minbpe and it took ~2173s per frame to tokenize. The whole "patchnization" took ~77.4s for a 20s video on my M2 Air.

The files in the repo work as follows:

1. patchnizer.py: Holds code for simple implementation of the three stages involved (extract_image_frames from video, reduce image_frames_to_patches of fixed sizes 16x16 pixels, then linearly_embed_patches into a 1D vector sequence with additional position embeddings.

2. patchnize.py: performs the whole process with custom configs (patch_size, created dirs, video - I am using the "dogs playing in snow" video by sora).

3. train.py: Trains the resulting one-dimensional vector sequence (linear_patch_embeddings + position_embeddings) on Karpathy's minbpe (a minimal implementation of the byte-pair encoding algorithm).

4. check.py: Checks if the patch embeddings match the original image patches by recovering the image frames from their corresponding image patches.

The Patchnizer class has three stubs:
- extract_image_frames() which chops the video (20sec) into 60 frames (i.e each frame is ~0.33 secs) each of size 1280x720 pixels (original video dims).
- image_frames_to_patches() which grids each image frame into 16x16 pixels tiles. This makes each frame has a total of 3600 image patches (i.e 80 rows by 45 columns).
- linearly_embed_patches() turns the image patches into patch embeddings (a long string of integers for each image patch) then adds a position embedding for each patch.

osanseviero femboysLover samusenps clem Lwasinam whooray cahlen


Stopwolf julien-c samusenps clem dillfrescott


Released updated weights for moondream2 today, with significantly improved benchmark scores!

https://huggingface.co/spaces/vikhyatk/moondream2

https://huggingface.co/vikhyatk/moondream2

Sylvestre osanseviero victor dim samusenps Dlbk clem Lewdiculous Tom-Neverwinter not-lain MoonRide


🚀 Introducing UltraTextbooks v2: The Ultimate Educational NLP Dataset! 📚

I've expanded the dataset to include an even wider range of high-quality textbooks, with a special focus on machine learning, mathematics, and coding. 💻🧮

With over 3 million examples and 6 GB of data, UltraTextbooks v2 is your go-to resource for training advanced language models and developing cutting-edge educational applications. 🎓

Explore the dataset on Hugging Face and unlock the power of AI in education! 🔓

https://huggingface.co/datasets/Locutusque/UltraTextbooks-2.0

osanseviero Felladrin clem samusenps Jjjjjo


Empower yourself with knowledge and skills. The power of knowledge is the key for advancement. Realize that without discovering your weaknesses, it would be hard to improve yourself. No one is perfect, no one knows everything, but we all have the freedom to learn and discover our true potential. It's important to surround yourself with positivity and people who help you push forward to improve. Lifelong learning is a crucial key of success.

#LifelongLearning #Success #Enthusiasm


Hi everyone,

I'd like to share our free GPT-4 chatbot: https://huggingface.co/spaces/yuntian-deng/ChatGPT4. Data collected from it will be shared back with the community in future releases of the WildChat dataset: https://huggingface.co/datasets/allenai/WildChat. Please help us reach 1 million conversations!

Thanks,
Yuntian

yuntian-deng iDrops jinghan23 ajibawa-2023 osanseviero samusenps Dlbk Tonic Lewdiculous psy-taha artghoul RobertRoss asp933 IdleIdiot SwedMlite Maria200035

artghoul Aerialyn lololololololokok caiquemoa Johanbb


Design2Code: How Far Are We From Automating Front-End Engineering? (2403.03163)

The "Design2Code: How Far Are We From Automating Front-End Engineering" paper presents a benchmark for multimodal large language models (LLMs) aimed at automating front-end web development by translating webpage designs (screenshots) into code. This task evaluates the models' ability to recreate webpages that are visually and structurally similar to the original designs.

Key Points:
* Introduces the Design2Code task and benchmark for converting webpage screenshots into code, aiming to automate front-end web development.
* Evaluates multimodal LLMs using comprehensive metrics for visual similarity and element matching.
* GPT-4V outperforms other models in terms of visual resemblance and content accuracy, with generated webpages often preferred over the original references.

Paper: https://huggingface.co/papers/2403.03163
Project page: https://salt-nlp.github.io/Design2Code/
Dataset: https://huggingface.co/datasets/SALT-NLP/Design2Code

Congrats to the authors for their work!


Learning a Decision Tree Algorithm with Transformers (2402.03774)

huggingface? X(formerly known as twitter)!

Yufan here, CS PhD student from UC San Diego.

LLMs as ML algorithms? see my latest work MetaTree: https://huggingface.co/papers/2402.03774

Read more:

https://evanzhuang.github.io/
https://twitter.com/yufan_zhuang

yzhuang qnguyen3 osanseviero victor samusenps shing3232 ZennyKenny


Wow! Hello world. How's ECCV submissions going?


Hi everyone!

Looking forward to engaging with you all 🤗


radames DmitryRyumin weizhey osanseviero stefan-it apol taesiri ZennyKenny den0620 samusenps edeani rahmanansari iv7dev Nacholmo erickdp EddyGiusepe jbilcke-hf leegao19 Noomam

rtscott2001 edeani CreitinGameplays jbilcke-hf


Sharing our paper and library for building LLM agent. The library is less than 1K code lines!
https://github.com/SalesforceAIResearch/AgentLite
https://arxiv.org/abs/2402.15538

clem osanseviero taesiri qgao007 victor samusenps kumarh1982 Hanyu66 theArif


osanseviero Suparious samusenps dashfunnydashdash ZennyKenny thomwolf Lewdiculous victor


👋 I'm Sam

I use ML and HPC to accelerate scientific discovery @ Argonne National Laboratory*

https://samforeman.me
https://twitter.com/saforem2
https://github.com/saforem2

* https://alcf.anl.gov/about/people/sam-foreman

osanseviero tomaarsen victor medmac01 mozayed


weizhey osanseviero tomaarsen MarinaraSpaghetti samusenps polodealvarado Lewdiculous HaiderSultanArc


Hi everyone,

I'd like to share our project on open-type Named Entity Recognition (NER). Our model uses a transformer encoder (BERT-like), making the computation overhead very minimal compared to use of LLMs. I've developed a demo that runs on CPU on Google Colab.

Colab Demo: https://colab.research.google.com/drive/1mhalKWzmfSTqMnR0wQBZvt9-ktTsATHB?usp=sharing

Code: https://github.com/urchade/GLiNER

Paper: https://arxiv.org/abs/2311.08526

clem NePe sgarbi osanseviero samusenps Tanvir1337 medmac01 merve ruman1 awinml maortal Antoine imihalcea richgjacksonaz Bailefan eek


DeepLearning.AI just announced a new short course: Open Source Models with Hugging Face 🤗, taught by Hugging Face's own Maria Khalusova, Marc Sun and Younes Belkada!

As many of you already know, Hugging Face has been a game changer by letting developers quickly grab any of hundreds of thousands of already-trained open source models to assemble into new applications. This course teaches you best practices for building this way, including how to search and choose among models.

You'll learn to use the Transformers library and walk through multiple models for text, audio, and image processing, including zero-shot image segmentation, zero-shot audio classification, and speech recognition. You'll also learn to use multimodal models for visual question answering, image search, and image captioning. Finally, you’ll learn how to demo what you build locally, on the cloud, or via an API using Gradio and Hugging Face Spaces.

Thank you very much to Hugging Face's wonderful team for working with us on this.

You can sign up for the course here: https://www.deeplearning.ai/short-courses/open-source-models-hugging-face/

osanseviero MariaK clem abidlabs Wauplin thomwolf Jofthomas kramp julien-c Violette philschmid VictorSanh lhoestq victor DmitryRyumin brunatrevelin m-ric loubnabnl samusenps leegao19 radames P3rry JAkinyemi qgao007 mvaloatto jeffboudier ybelkada hysts neerajjulka lysandre linoyts muhtasham madroid marcsun13 keerekeerweere OfferL MarinaraSpaghetti bulentsiyah dijvar pankaj-munde medmac01 beenakurian

clem abidlabs Wauplin thomwolf osanseviero Jofthomas julien-c Violette philschmid lhoestq victor katielink loubnabnl samusenps rowbradley hmb andrewrreed radames rtscott2001 yjernite jeffboudier ybelkada lysandre muhtasham madroid marcsun13 quibblerquery dimbyTa mayacinka

osanseviero clem abidlabs Wauplin thomwolf Jofthomas julien-c Violette philschmid VictorSanh lhoestq victor brunatrevelin m-ric loubnabnl radames ybelkada lysandre muhtasham marcsun13


Design2Code

How Far Are We From Automating Front-End Engineering?

https://huggingface.co/papers/2403.03163

Generative AI has made rapid advancements in recent years, achieving unprecedented capabilities in multimodal understanding and code generation. This can enable a new paradigm of front-end development, in which multimodal LLMs might directly convert visual designs into code implementations. In this work, we formalize this as a Design2Code task and conduct comprehensive benchmarking. Specifically, we manually curate a benchmark of 484 diverse real-world webpages as test cases and develop a set of automatic evaluation metrics to assess how well current multimodal LLMs can generate the code implementations that directly render into the given reference webpages, given the screenshots as input. We also complement automatic metrics with comprehensive human evaluations. We develop a suite of multimodal prompting methods and show their effectiveness on GPT-4V and Gemini Pro Vision. We further finetune an open-source Design2Code-18B model that successfully matches the performance of Gemini Pro Vision. Both human evaluation and automatic metrics show that GPT-4V performs the best on this task compared to other models. Moreover, annotators think GPT-4V generated webpages can replace the original reference webpages in 49% of cases in terms of visual appearance and content; and perhaps surprisingly, in 64% of cases GPT-4V generated webpages are considered better than the original reference webpages. Our fine-grained break-down metrics indicate that open-source models mostly lag in recalling visual elements from the input webpages and in generating correct layout designs, while aspects like text content and coloring can be drastically improved with proper finetuning.

victor DavidVivancos hmb vladbogo clem TheSaifurRahman sgarbi krishnapraveen samusenps jnopareboateng seanmiranda julien-c awacke1 marcelovicentegc Ananze


Hey, it took some time but I finally moved out and got internet back, so here I am again!
A lot of things to get updated on, I will try to reply to each of you ASAP.
See you soon!

kawaiibastard RUcrittingME Ainonake MarinaraSpaghetti victor okeksama papercanteen111 saishf WaefreBeorn eevee32x antiven0m 010O11 damerajee CarlosIABn samusenps AtonMountlook krisshen Zeldazackman Diavator Adriato wath5 ConquestofElysium jarjarbinks SGNghiax SerialKicked ClaudioItaly WilsonWan IHaBiS locke12 WeForgot Shadowplague Tillx85

mayurninama bluuwhale ZZnake rywiz Diavator cpham9000 xpgx1 AtonMountlook frgthzjmk


hey there folks new Yi model just came out , and i had a gradio interface ready since their last releases.

it's just a base model but you can check it out here : https://huggingface.co/spaces/Tonic/Yi-9B

cant wait to fine tune this one 🤗🚀


You can now use gpt4all.io to instantly search, download, and chat with models hosted on huggingface!

osanseviero victor diegotluz samusenps ybelkada Tonic pcuenq ajibawa-2023 davanstrien clem zpn EtienneDosSantos Nymbo

osanseviero samusenps ybelkada alielfilali01 pcuenq clem Hugorowan zpn PeepDaSlan9 Nymbo


SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis (2311.17590)

🚀😈🌟 New Research Alert - CVPR 2024! 🌟😈 🚀
📄 Title: SyncTalk: The Devil 😈 is in the Synchronization for Talking Head Synthesis 🌟🚀

📝 Description: SyncTalk synthesizes synchronized talking head videos, employing tri-plane hash representations to maintain subject identity. It can generate synchronized lip movements, facial expressions, and stable head poses, and restores hair details to create high-resolution videos.

👥 Authors: Ziqiao Peng et al.

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2311.17590

🔗 Github Page: https://ziqiaopeng.github.io/synctalk
🔗 Repository: https://github.com/ZiqiaoPeng/SyncTalk

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #TalkingHeads #Synthesis #TriPlaneHash #FacialExpressions #LipSyncing #HighResolutionVideos #CVPR2024 #DeepLearning #Animation #Innovation

DmitryRyumin osanseviero samusenps minhdang Tonic takeraparterer brainhome MiSTe-R JasonZJ victor vladbogo clem dashfunnydashdash lvalue


Diaries of Open Source. Part 2. Open Source is going brrrrr

🚀The European Space Agency releases MajorTOM, a dataset of earth observation covering half the earth. The dataset has 2.5 trillion pixels! Congrats @aliFrancis and @mikonvergence !
Dataset: https://huggingface.co/datasets/Major-TOM/Core-S2L2A
Viewer: https://huggingface.co/spaces/Major-TOM/MajorTOM-Core-Viewer

🍞Re-ranking models by MixedBreadAI, with very high quality, Apache 2 license, and easy to use!
Models: https://huggingface.co/models?other=reranker&sort=trending&search=mixedbread-ai
Blog: https://www.mixedbread.ai/blog/mxbai-rerank-v1

🧊StabilityAI and TripoAI release TripoSR, a super-fast MIT-licensed image-to-3D model!
Model: https://huggingface.co/stabilityai/TripoSR
Demo: https://huggingface.co/spaces/stabilityai/TripoSR

🤝Together AI and HazyResearch release Based
Models and datasets: https://huggingface.co/collections/hazyresearch/based-65d77fb76f9c813c8b94339c
GH repo: https://github.com/HazyResearch/based

🌊LaVague: an open-source pipeline to turn natural language into browser actions! It can run locally with https://huggingface.co/HuggingFaceH4/zephyr-7b-gemma-v0.1
Read more about it at https://huggingface.co/posts/dhuynh95/717319217106504

🏆Berkeley Function-Calling Leaderboard
Read about it: https://gorilla.cs.berkeley.edu/blogs/8_berkeley_function_calling_leaderboard.html
Leaderboard: https://gorilla.cs.berkeley.edu/leaderboard.html

🐬Sailor-Chat: chat models built on top of OpenOrca and @sarahooker CohereForAI Aya project. They can be used for South-East Asia languages such as Indonesian, Thai, Vietnamese, Malay and Lao!
Models: https://huggingface.co/collections/sail/sailor-language-models-65e19a749f978976f1959825
Demo: https://huggingface.co/spaces/sail/Sailor-7B-Chat

🤗Arabic-OpenHermes-2.5: OpenHermes dataset translated to Arabic https://huggingface.co/datasets/2A2I/Arabic-OpenHermes-2.5

See the previous part here https://huggingface.co/posts/osanseviero/622788932781684

mrm8488 mikonvergence peterschmidt85 DmitryRyumin yjernite SivilTaram zirui3 ajibawa-2023 davanstrien vladbogo clem irenesolaiman mvaloatto Jose7juanFdz kramp juliuslipp

kshitizkhanal7 samusenps SivilTaram clefourrier lucabaggi clem irenesolaiman nisten


AtomoVideo: High Fidelity Image-to-Video Generation (2403.01800)

AtomoVideo

High Fidelity Image-to-Video Generation

https://huggingface.co/papers/2403.01800

Recently, video generation has achieved significant rapid development based on superior text-to-image generation techniques. In this work, we propose a high fidelity framework for image-to-video generation, named AtomoVideo. Based on multi-granularity image injection, we achieve higher fidelity of the generated video to the given image. In addition, thanks to high quality datasets and training strategies, we achieve greater motion intensity while maintaining superior temporal consistency and stability. Our architecture extends flexibly to the video frame prediction task, enabling long sequence prediction through iterative generation. Furthermore, due to the design of adapter training, our approach can be well combined with existing personalised models and controllable modules.


We are thrilled to announce the release of the OmniACT dataset!  This revolutionary dataset and benchmark focuses on pushing the limits of how virtual agents can facilitate the automation of our computer tasks. Imagine less clicking and typing, and more observation as your computer takes care of tasks such as organizing schedules or arranging travel arrangements on its own.

Check it out ➡ [OmniACT Dataset on Hugging Face](https://huggingface.co/datasets/Writer/omniact)

For a deep dive, here’s the paper: [OmniACT Paper](https://arxiv.org/abs/2402.17553)

melisa osanseviero victor nbroad samusenps jeffboudier julien-c kiranr lhoestq pierrci nikjain clem NERDDISCO bk-ignite medmac01 thomwolf sbrandeis

femboysLover victor M1cler julien-c lhoestq clem bk-ignite medmac01 thomwolf


🔥 New multimodal leaderboard on the hub: ConTextual!

Many situations require models to parse images containing text: maps, web pages, real world pictures, memes, ... 🖼
So how do you evaluate performance on this task?

The ConTextual team introduced a brand new dataset of instructions and images, to test LMM (large multimodal models) reasoning capabilities, and an associated leaderboard (with a private test set).

This is super exciting imo because it has the potential to be a good benchmark both for multimodal models and for assistants' vision capabilities, thanks to the instructions in the dataset.

Congrats to @rohan598, @hbXNov, @kaiweichang and @violetpeng !!

Learn more in the blog: https://huggingface.co/blog/leaderboard-contextual
Leaderboard: https://huggingface.co/spaces/ucla-contextual/contextual_leaderboard

vladbogo BrigitteTousi Kukedlc osanseviero victor VictorSanh samusenps JasonZJ mvaloatto rohan598 puneetjindalisb whitew1994


VisionLLaMA: A Unified LLaMA Interface for Vision Tasks (2403.00522)

VisionLLaMA is a new vision transformer architecture that adapts the successful LLaMA language model design for vision tasks. By integrating components like rotary positional embeddings, SwiGLU activation, and LayerNorm from LLaMA, VisionLLaMA achieves very promising performance across various vision tasks, including image generation, classification, semantic segmentation, and object detection.

Keypoints:
* Outperforms state-of-the-art vision transformers like DiT, SiT, DeiT3, and Swin on multiple benchmarks and tasks.
* Leverages Auto-Scaled 2D Rotary Positional Embeddings (AS2DRoPE) to handle variable input resolutions efficiently.
* Serves as a powerful, unified modeling framework for vision generation and understanding tasks.

Paper: https://huggingface.co/papers/2403.00522
GitHub repo: https://github.com/Meituan-AutoML/VisionLLaMA

Congrats to the authors for their work!

clem jucamohedano osanseviero victor samusenps sgarbi MexIvanov trysem


Robust Speech Recognition via Large-Scale Weak Supervision (2212.04356)

Why does returning timestamps help Whisper reduce hallucinations? 🧐

Empirically, most practitioners have found that setting `return_timestamps=True` helps reduce hallucinations, particularly when doing long-form evaluation with Transformers’ “chunked” algorithm.

But why does this work?..

My interpretation is that forcing the model to predict timestamps is contradictory to hallucinations. Suppose you have the transcription:
```markdown
The cat sat on the on the on the mat.
```
Where we have a repeated hallucination for “on the”. If we ask the model to predict timestamps, then the “on the” has to contribute to the overall segment-level timing, e.g.:
```markdown
<|0.00|> The cat sat on the on the on the mat.<|5.02|>
```
However, it’s impossible to fit 3 copies of “on the” within the time allocation given to the segment, so the probability for this hallucinatory sequence becomes lower, and the model actually predicts the correct transcription with highest probability:
```markdown
<|0.00|> The cat sat on the mat.<|5.02|>
```
In this sense, the end timestamp is of the opposite of the initial timestamp constraint they describe in Section 4.5 of the paper https://huggingface.co/papers/2212.04356 → it helps the model remove extra words at the end of the sequence (rather than the initial timestamp which helps when the model ignores words at the start), but the overall principle is the same (using timestamps to improve the probability of more realistic sequences).

Leaving it open to you: why do you think timestamps reduces Whisper hallucinations?

clem abellion vladbogo daedaluswave osanseviero samusenps Csplk DRRRRDFFD on1onmangoes sanchit-gandhi jayzhuo michaelsymph EBalykov Mutturaj DattaKiran OjciecTadeusz ericlaycock dkakaie WIldOni ayoalfonso dzlandis peterciank


What if you could casually access your remote GPU in HF Spaces from the comfort of your local VSCode 🤯

ajibawa-2023 Gabriel victor mvaloatto medmac01 clem BrigitteTousi musfiqdehan osanseviero asusevski samusenps cnmoro mathiasn1 JackCloudman Nadav-Timor Felladrin muhtasham dwidlee peterschroedl not-lain radames

fffiloni taufiqdp medmac01 natolambert clem BrigitteTousi umuthopeyildirim osanseviero vtiyyal1 xxXWarMachineRoXxx muhtasham peterschroedl kramp not-lain seyf1elislam radames

tuanlda78202 medmac01 clem BrigitteTousi muhtasham peterschroedl not-lain


Huge new dataset for remote sensing on HF: Major TOM

More info in this thread https://x.com/mikonvergence/status/1764913256102031402?s=20

Note: video coming soon

mikonvergence abbasm2 clem osanseviero samusenps dillfrescott


Bellman, the Swedish finetune, has once again returned in his biggest incarnation yet, at 12b. Based on Mistral-Nemo-Instruct: https://huggingface.co/neph1/Mistral-Nemo-Instruct-bellman-12b


Spent the weekend testing out some prompts with 🕵♂Mystery Bot🕵♂ on my mobile... exciting things are coming soon for the following languages:

🌐Arabic, Chinese, Czech, Dutch, English French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Persian, Polish, Portuguese, Romanian, Russian, Spanish, Turkish, Ukrainian, and Vietnamese!🌐


I’m recently experimenting with the Flux-Ultra Realism and Real Anime LoRA models, using the Flux.1-dev model as the base. The model and its demo example are provided in the Flux LoRA DLC collections.📃

🥳Demo : 🔗 https://huggingface.co/spaces/prithivMLmods/FLUX-LoRA-DLC

🥳Model:
-  https://huggingface.co/prithivMLmods/Canopus-LoRA-Flux-UltraRealism-2.0
-  https://huggingface.co/prithivMLmods/Flux-Dev-Real-Anime-LoRA

🥳For more details, please visit the README.md of the Flux LoRA DLC Space & https://huggingface.co/collections/prithivMLmods/lora-space-collections-6714b72e0d49e1c97fbd6a32

Minhajameen Stopwolf AdinaY prithivMLmods ai4life44


🌐 Introducing Websim.ai User Projects Dataset - https://huggingface.co/datasets/nyuuzyou/websim

Dataset highlights:
- 137,452 user projects from Websim.ai, a service for creating small sites using Large Language Models (LLMs)
- Primarily in English, with potential for multilingual content in generated websites
- Each entry includes: project metadata, user information, and generated HTML content
- Contains detailed information about project revisions, site generation, and user interactions
- Data covers a wide range of user-generated website projects created through AI assistance
- Dedicated to the public domain under Creative Commons Zero (CC0) license

The dataset can be used for analyzing AI-assisted web development trends, studying user behavior in LLM-powered creative tools, and exploring the capabilities of language models in web design.


This is probably a very hot take, but here goes nothing.

With the incredibly accurate LoRAs we see emerge for high quality models like FLUX from services like fal.ai that offer training within single digit minutes, e.g. 2 min per 1000 iterations.

Why the hell are people publishing private LoRAs as public models?!
Take a look at this listing: https://huggingface.co/models?other=base_model:adapter:black-forest-labs%2FFLUX.1-dev&sort=created

I would expect that people that hold a HF account have some kind of forward thinking. Heck, do you really want to give anyone the power to create ultra realistic images of yourself?!

Didn't we learn anything from social media?
I am puzzled..


Last Week in Medical AI: Top LLM Research Papers/Models 🔥
🏅 (October 12 - October 19, 2024)

Medical LLM & Other Models:
- OLAPH: Factual Biomedical LLM QA
- LLMD: Interpreting Longitudinal Medical Records
- LifeGPT: Generative Transformer for Cells
- MedCare: Decoupled Clinical LLM Alignment
- Y-Mol: Biomedical LLM for Drug Development

Frameworks and Methodologies:
- MedINST: Biomedical Instructions Meta Dataset
- Democratizing Medical LLMs via Language Experts
- MCQG-SRefine: Iterative Question Generation
- Adaptive Medical Language Agents
- MeNTi: Medical LLM with Nested Tools

Medical LLM Applications:
- AGENTiGraph: LLM Chatbots with Private Data
- MMed-RAG: Multimodal Medical RAG System
- Medical Graph RAG: Safe LLM via Retrieval
- MedAide: Multi-Agent Medical LLM Collaboration
- Synthetic Clinical Trial Generation

Medical LLMs & Benchmarks:
- WorldMedQA-V: Multimodal Medical LLM Dataset
- HEALTH-PARIKSHA: RAG Models Evaluation
- Synthetic Data for Medical Vision-Language

Now you can watch and listen to the latest Medical AI papers daily on our YouTube and Spotify channels as well!

- Youtube: https://youtu.be/LROOjWXUgvg?si=s-nNDOSD3BrsHYjQ
- Spotify : https://open.spotify.com/episode/12xeN2vnOTRdDrHbWqhV6I?si=bd7c8d9fee8049fd

aaditya benhachem Joseph717171 shetumohanto


How the 🗣🏆 leaderboard of a merged TTS Arena with the 🤗 Spaces fork would look like. These results are somewhat unreliable as some models have not challenged the other in the list. And the original TTS Arena used only narration type sentences.


🚀  Announcement for the Lovely community! 🚀

Just launched the https://huggingface.co/spaces/zamal/DeepSeek-VL-1.3B-Chat on Hugging Face, and it's ready for YOU to explore! 💬🖼

This full-fledged model is perfect for advanced image and text interactions, with zero GPU required. The Deepseek VL-1.3B Chat typically needs around 8 GB of VRAM and storage of almost 4 GB, but now you can experience it hassle-free right on our space!

Want something lighter? We’ve also uploaded a 4 bit quantized version (just around 1GB!), available on my profile. Perfect for those with limited hardware. 🌍🔍

Come try it now and see what this model can do! 🚀✨


Triton nanoGPT now has a custom cross entropy loss kernel 🚀
Next: matmul, gradually overthrowing all major PyTorch ops:)

Simplified pseudo for parallel cross-entropy loss compute:
- init program: get pid, compute offsets, load targets.
- init row_max and row_sum.
- for-loop1 (find max logits): update row_max with max logits.
- for-loop2 (compute softmax and loss): compute row_sum, update loss.
- add log(row_sum) and store loss.

Code: https://github.com/Jaykef/ai-algorithms/blob/main/triton_nanoGPT.ipynb


The dataset of all Italian laws  just released https://huggingface.co/datasets/mii-llm/gazzetta-ufficiale

Kudos to @efederici @mferraretto @z-uo


The Stable Diffusion 3 research paper broken down, including some overlooked details! 📝

Model
📏 2 base model variants mentioned: 2B and 8B sizes

📐 New architecture in all abstraction levels:
- 🔽 UNet; ⬆ Multimodal Diffusion Transformer, bye cross attention 👋
- 🆕 Rectified flows for the diffusion process
- 🧩 Still a Latent Diffusion Model

📄 3 text-encoders: 2 CLIPs, one T5-XXL; plug-and-play: removing the larger one maintains competitiveness

🗃 Dataset was deduplicated with SSCD which helped with memorization (no more details about the dataset tho)

Variants
🔁 A DPO fine-tuned model showed great improvement in prompt understanding and aesthetics
✏ An Instruct Edit 2B model was trained, and learned how to do text-replacement

Results
✅ State of the art in automated evals for composition and prompt understanding
✅ Best win rate in human preference evaluation for prompt understanding, aesthetics and typography (missing some details on how many participants and the design of the experiment)

Paper: https://stabilityai-public-packages.s3.us-west-2.amazonaws.com/Stable+Diffusion+3+Paper.pdf

osanseviero fffiloni victor mvaloatto clem BrigitteTousi WaveCut EtienneDosSantos samusenps Tonic chethan62 Eungbean MrForExample Obhee GPT007

clem BrigitteTousi samusenps leegao19 Tonic Bils gonduras


Major TOM: Expandable Datasets for Earth Observation (2402.12095)

🗺 Major TOM: Expandable Datasets for Earth Observation

🚨 RECORD-BREAKING EO DATASET: the largest ever ML-ready Sentinel-2 dataset! It covers almost every single point on Earth captured by the Copernicus Sentinel-2 satellite. @mikonvergence and I are thrilled to finally announce the release of https://huggingface.co/datasets/Major-TOM/Core-S2L2A and https://huggingface.co/datasets/Major-TOM/Core-S2L1C

🌍 About half of the entire planet is covered. That's 2,245,886 patches of 1068 x 1068 pixels, available in both L1C and L2A. At 10 m resolution, we've got 256 million square km with over 2.5 trillion pixels. It's all yours with a few lines of code. See the paper linked below 🔽 for more info!

🧱 And this is just the beginning. We are currently preparing more datasets from different satellites for the Major TOM org. TOM stands for Terrestrial Observation Metaset - a simple set of rules for building an ecosystem of ML-ready EO datasets, which can be seamlessly combined as if they were Lego bricks.

🚴♀ Want to take the dataset for a spin? We have a viewer app on spaces that lets you go anywhere on Earth and shows you the data, if its available https://huggingface.co/spaces/Major-TOM/MajorTOM-Core-Viewer

📰 Preprint paper: https://huggingface.co/papers/2402.12095
💻 Colab example: https://colab.research.google.com/github/ESA-PhiLab/Major-TOM/blob/main/03-Filtering-in-Colab.ipynb

Thank you to the amazing 🤗Hugging Face team for the support on this one! @osanseviero @lhoestq @BrigitteTousi

mikonvergence lhoestq osanseviero julien-c taufiqdp mvaloatto clem BrigitteTousi nicolasdec kramp csaybar

lhoestq osanseviero julien-c clem BrigitteTousi nicolasdec mrapplegate samusenps brunosan


Just released moondream2 - a small 1.8B parameter vision language model. Now fully open source (Apache 2.0) so you can use it without restrictions on commercial use!

https://huggingface.co/vikhyatk/moondream2

osanseviero mrfakename andysalerno natolambert ajibawa-2023 Vigilence vignesh-fynd Tonic Csplk DmitryRyumin Chris4K chansung Dlbk Yoben fffiloni Kukedlc taufiqdp clem ostris samusenps saikatkumardey multimodalart veravira euclaise twenkid sbrandeis

osanseviero andysalerno OsnNos Vigilence vignesh-fynd Tonic kramp Dlbk sourceoftruthdata clem t1u1 damerajee


GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single
  Video via Animatable 3D Gaussians (2312.02134)

🚀🎬🌟 New Research Alert - CVPR 2024! 🌟🎬 🚀
📄 Title: GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians 🌟🚀

👥 Authors: Liangxiao Hu et al.

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2312.02134

🔗 Github Page: https://huliangxiao.github.io/GaussianAvatar
🔗 Repository: https://github.com/huliangxiao/GaussianAvatar

🔗 Video: https://www.youtube.com/watch?v=a4g8Z9nCF-k

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #GaussianAvatar #3DGaussians #HumanAvatarModeling #PoseDependentAppearance #DynamicAppearanceModeling #MotionEstimation #MonocularSettings #AppearanceQuality #RenderingEfficiency #CVPR2024 #DeepLearning #Animation #Innovation

DmitryRyumin vladbogo mickdarling clem MiSTe-R StrangeSX szymonrucinski Tonic sbrandeis


🌊 Released #LaVague, fullly open-source AI pipeline to turn natural language into browser actions!

In less than 150 lines of code (RAG with local embedding + Zephyr-7b-Gemma locally or Mixtral on HF Inference API), it generates #Selenium code from user query. In this GIF you can see it follow user instructions to command a browser to browse HF website!

Try it on Colab: colab.research.google.com/github/dhuynh95/LaVague/blob/main/LaVague.ipynb
GitHub: github.com/dhuynh95/LaVague

Pretty exciting how it becomes possible to create an AI assistant that could perform actions for us, such as logging on gov accounts, fill forms, or pull personal information!

It was quite fun to hack in the weekend using open-source tools, from @huggingface local embedding with transformers for local inference or HF Inference API, to RAG with @llama_index, through @MistralAI Mixtral model!

Some challenges: to make it run on Colab for the #GPU Poors, I first resorted to @huggingface Inference API with Mixtral as it was the only model good enough (gemma-7b did not make it and refused to produce code). But after some experimentations, I managed to make it work a local Zephyr-7b-Gemma so that people could run this assistant fully locally!

Because I used an off-the-shelf model, I had to improve performance with few-shot learning and Chain Of Thought, which managed to generate appropriate code!

I hope this project will herald a new dawn where transparent, private and local AI assistants help automate menial but critical tasks, such as helping fill taxes, book accomodation, or research information for us.

osanseviero kramp ejsellers vladbogo mexicanamerican 0xKrypt0r Dlbk qnguyen3 julien-c pranay-j victor diwank iamrobotbear samusenps bruceunx George-Blaze superLk timnon

osanseviero VictorSanh Guilherme34 Dlbk julien-c victor clem diwank bonsaielectric iamrobotbear samusenps z3ymd kgawron bruceunx Tavi31 superLk sbrandeis

fffiloni victor clem iamrobotbear Tavi31


Understand research papers easier with automatically generated Q&As by LLM (Gemini 1.0 Pro). For this purpose, I have built two projects.

- [Auto Paper Analysis](https://github.com/deep-diver/auto-paper-analysis) let you generate QAs on a list of papers. The paper list could be specified either from Hugging Face's Daily Papers or in a set of raw arXiv IDs. Then the generated QA dataset could be pushed to the Hugging Face Dataset. Refer to the attached image.

- [PaperQA Space application](https://huggingface.co/spaces/chansung/paper_qa) shows how to interact with the generated QA dataset. Search the paper by keyword or date, then understand it with the QAs (in ELI5 and technical versions). Check out the attached video, or visit the space directly.

This is a baby step for the automated paper analysis (summarization) to easily consume the exploding information in the field of AI. In the next phase, I am gonna need spend my time to enhance prompt engineering, UI/UX (such as Like/Dislike system), ...

However, in the meantime, I hope this project could be helpful for someone who struggles on understanding papers (new papers comes out even when I did finish reading a paper from yesterday yet,,)!

Also, any suggestion to improve this, please let me know :)

chansung Kukedlc medmac01 vladbogo arcdyn Lewdiculous pranay-j Srulikbd samusenps bh-lee akhaliq boapps genaiLLM sbrandeis


VisionLLaMA

A Unified LLaMA Interface for Vision Tasks

https://huggingface.co/papers/2403.00522

Large language models are built on top of a transformer-based architecture to process textual inputs. For example, the LLaMA stands out among many open-source implementations. Can the same transformer be used to process 2D images? In this paper, we answer this question by unveiling a LLaMA-like vision transformer in plain and pyramid forms, termed VisionLLaMA, which is tailored for this purpose. VisionLLaMA is a unified and generic modelling framework for solving most vision tasks. We extensively evaluate its effectiveness using typical pre-training paradigms in a good portion of downstream tasks of image perception and especially image generation. In many cases, VisionLLaMA have exhibited substantial gains over the previous state-of-the-art vision transformers. We believe that VisionLLaMA can serve as a strong new baseline model for vision generation and understanding.

osanseviero clem Kukedlc hasangoni vladbogo notune Yoben mvaloatto TD788432 samusenps Abecid jsfs11 Ryukijano sprime01 researchase sbrandeis


Today, we're launching an effort to empower the community to build impactful datasets collectively.

Good data is essential for the open-source AI community. Recently, Argilla and Hugging Face launched Data is Better Together. In less than two weeks, over 350 people ranked over 10k  prompts.

Today, we're shifting our focus to help support other community efforts to create datasets using Argilla and Hugging Face Spaces. This workflow means anyone with a Hugging Face account can contribute to a dataset in less than a minute.  We want to hear from anyone with ideas for creating important datasets as a community. This could include things like:

- Creating preference data for a language that lacks high-quality preference datasets.
- Building evaluation datasets for a new domain.
- Developing a dataset for a new task.

If you would like to get involved, join us in the `#data-is-better-together` Discord channel: https://discord.com/channels/879548962464493619/1205128865735770142.

You can read more in this blog post from @dvilasuero and I: https://huggingface.co/blog/community-datasets

osanseviero clem Kukedlc ajibawa-2023 danielus julien-c samusenps dillfrescott


🚨 Now you can run Starcoder- 2 models locally on your Mac M1 Pro Apple Silicon with 16GB memory! 🧑🏽💻 ⚡✨

Below is the UX with Twinny extension using https://huggingface.co/bigcode/starcoder2-3b for FIM and https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf for chat. Dev tools is showing the prompt being sent to ollama server.

Starcoder-2 is now supported in `llama.cpp`  https://github.com/ggerganov/llama.cpp/pull/5795!
```
cd llama.cpp
python convert-hf-to-gguf.py ../starcoder2-3b/ --outfile models/starcoder2-3b.gguf --outtype "f16"
./quantize models/starcoder2-3b.gguf models/starcoder2-3b-Q4_K_M.gguf Q4_K_M
```
For more details, please go through the following tweet thread: https://x.com/sourab_m/status/1764583139798823235?s=20

osanseviero lvwerra clem Kukedlc notune jshuadvd codito rjmacarthy

clem Kukedlc Yoben BrigitteTousi rjmacarthy


AtP*: An efficient and scalable method for localizing LLM behaviour to
  components (2403.00745)

🔍 Today's pick in Interpretability & Analysis of LMs: AtP*: An efficient and scalable method for localizing LLM behaviour to components by J. Kramár T. Lieberum R. Shah @NeelNanda

The attribution patching method (AtP) can provide fast and effective approximations of activation patching, requiring only two forward passes and one backward pass to estimate the contribution of all network components for a given prompt pair.

While previous work highlighted the effectiveness of attribution patching, authors identify two settings leading to false negatives using AtP:

- When estimating the contribution of pre-activation components, if clean and noise inputs don’t lie in the same activation region, the first-order gradient approximation provided by the gradient leads to large errors (Fig 3).
- When the sum of direct and indirect effects is close to 0, even small approximation errors introduced by nonlinearities can greatly  affect the estimated contribution.

Authors propose two changes to the AtP method to mitigate such issues:

- recomputing the attention softmax for the selected component, and then taking a linear approximation to the remaining part of the model (QK Fix)
- Iteratively zeroing gradients at layers contributing to the indirect effects causing cancellation (GradDrop)

AtP and AtP* are compared across several patching settings for Pythia models, finding them effective while much less computationally expensive than other approaches. A new methodology is also proposed to estimate the magnitude of AtP* false negatives given a set of samples and desired confidence levels.

📄 Paper: https://huggingface.co/papers/2403.00745

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9

osanseviero NeelNanda clem Kukedlc julien-c nirmalendu01


Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers (2402.19479)

Panda-70M is a new large-scale video dataset comprising 70 million high-quality video clips, each paired with textual captions, designed to be used as pre-training for video understanding tasks.

Key Points:
* Automatic Caption Generation: Utilizes an automatic pipeline with multiple cross-modality teacher models to generate captions for video clips.
* Fine-tuned Caption Selection: Employs a fine-tuned retrieval model to select the most appropriate caption from multiple candidates for each video clip.
* Improved Performance: Pre-training on Panda-70M shows significant performance gains in video captioning, text-video retrieval, and text-driven video generation.

Paper: https://huggingface.co/papers/2402.19479
Project page: https://snap-research.github.io/Panda-70M/
Code: https://github.com/snap-research/Panda-70M

Congrats to the authors @tschen, @aliaksandr-siarohin et al. for their work!

toanvu upleader ajibawa-2023 clem Kernel osanseviero dkgee ankur296 ChavyvAkvar tschen

clem osanseviero tschen nisten sbrandeis


Learning to Generate Instruction Tuning Datasets for Zero-Shot Task
  Adaptation (2402.18334)

Diaries of Open Source. Part 1.

What a week! Here are some of the exciting Open Source releases of the week!

1. BigCode releases The Stack v2 and StarCoder 2
Resources in https://huggingface.co/posts/loubnabnl/596860170283496
Blog https://huggingface.co/blog/starcoder2
Collection: https://huggingface.co/collections/bigcode/starcoder2-65de6da6e87db3383572be1a

2. Playground v2.5, a very powerful new text-to-image model
Model: https://huggingface.co/playgroundai/playground-v2.5-1024px-aesthetic
Demo: https://huggingface.co/spaces/playgroundai/playground-v2.5
Blog: https://playground.com/blog/playground-v2-5

3.Evo: DNA foundation models
Blog: https://arcinstitute.org/news/blog/evo
Models: https://huggingface.co/togethercomputer/evo-1-131k-base

4. OpenHermesPreferences: a dataset of ~1 million AI Preferences https://huggingface.co/datasets/argilla/OpenHermesPreferences

5. SpeechBrain 1.0: a toolkit with hundreds of recipes and pretrained models for audio-related tasks, such as speech recognition, diarization, and enhancement. New major release!
HF repos: https://huggingface.co/speechbrain
Website: https://speechbrain.github.io/

6. Tower: a suite of Llama-based multilingual translation models  https://huggingface.co/collections/Unbabel/tower-659eaedfe36e6dd29eb1805c

7. AllenAI releases OLMo-7B-Instruct
 https://huggingface.co/collections/allenai/olmo-suite-65aeaae8fe5b6b2122b46778

8. DIBT - An crowdsourced effort to human-rate prompts. Its 10k prompts dataset is released ttps://huggingface.co/datasets/DIBT/10k_prompts_ranked

9. ChatMusician: A Llama 2 fine-tuned model for music generation  https://huggingface.co/m-a-p/ChatMusician

10. Bonito, an model that converts data into synthetic instruction datasets
GitHub: https://github.com/BatsResearch/bonito
Model: https://huggingface.co/BatsResearch/bonito-v1
Paper: https://huggingface.co/papers/2402.18334

DmitryRyumin vladbogo clem chansung Kernel kramp tofuCheng tomaarsen lvwerra dkgee not-lain julien-c mvaloatto


MagicAnimate: Temporally Consistent Human Image Animation using
  Diffusion Model (2311.16498)

🚀💃🌟 New Research Alert - CVPR 2024! 🌟🕺 🚀
📄 Title: MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model 🌟🚀

👥 Authors: @junhao910323, @hansyan et al.

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🤗 Demo: https://huggingface.co/spaces/zcxu-eric/magicanimate

🔗 Paper: https://huggingface.co/papers/2311.16498
🔗 Github Page: https://showlab.github.io/magicanimate/
🔗 Repository: https://github.com/magic-research/magic-animate

🔥 Model 🤖: https://huggingface.co/zcxu-eric/MagicAnimate

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #MagicAnimate #DiffusionModel #HumanImageAnimation #CVPR2024 #Diffusion #DeepLearning #Innovation

DmitryRyumin musfiqdehan pkedzia vladbogo osanseviero victor toanvu clem samusenps sbrandeis


What Evidence Do Language Models Find Convincing? (2402.11782)

"What Evidence Do Language Models Find Convincing?" is a new paper that explores what types of evidence and argumentation techniques language models find convincing when presented with ambiguous, open-domain questions that have conflicting answers online.

Keypoints:
* Dataset: It introduces "ConflictingQA," a dataset of controversial questions and real-world evidence paragraphs supporting both "yes" and "no" answers.
* Convincingness Metric: It uses the "paragraph win rate" - when shown two conflicting paragraphs, this measures how often a model predicts the answer that aligns with a given paragraph's stance.
* Current models rely on the relevance of the content to the query, while largely ignoring stylistic features such as whether a text contains scientific references or if it is written with a neutral tone.

Congrats to the authors for their work!

Paper: https://huggingface.co/papers/2402.11782
Code: https://github.com/AlexWan0/rag-convincingness

samusenps dillfrescott osanseviero gsarti victor clem Kukedlc victorshenbr sbrandeis


Introducing: Zephyr Gemma!

The community has struggled to do a good preference-tune of Gemma, so the amazing @lewtun and @philschmid built an open-source recipe and trained a model to help people get started.

Handbook: https://github.com/huggingface/alignment-handbook/blob/main/recipes/zephyr-7b-gemma/README.md
Model: https://huggingface.co/HuggingFaceH4/zephyr-7b-gemma-v0.1
Demo: https://huggingface.co/spaces/HuggingFaceH4/zephyr-7b-gemma-chat

Some interesting details
- Fine-tuned on DEITA and DPOed with Argilla DPO dataset
- Very strong MT Bench results (7.81), better than Zephyr Beta (mistral based) and Gemma Instruct
- Can run locally with tools such as llama.cpp on a Mac
- Not so good AGIEval results compared to mistral-based tunes
- All training code is open-sourced
- Trained for 105 minutes on 8x H100
- No system message

Big kudos to the team! Super exciting to see a good fine-tune for Gemma

yjernite vladbogo clem samusenps dkyazze taufiqdp pdina


It feels awkward having my first post sharing my stuff, but this is a weekend project that I really enjoyed working on. I'd love to meet more people interested in random ideas like this.

A hard part of building AI applications is choosing which model to use. What if we don’t have to? What if we can predict the best model for any prompt?

Predictive human preference aims to predict which model users might prefer for a specific query.

https://huyenchip.com/2024/02/28/predictive-human-preference.html

One use case is model routing. If we know in advance that for a prompt, users will prefer Claude Instant’s response over GPT-4, and Claude Instant is cheaper/faster than GPT-4, we can route this prompt to Claude Instant. Model routing has the potential to increase response quality while reducing costs and latency.

One pattern is that for simple prompts, weak models can do (nearly) as well as strong models. For more challenging prompts, however, users are more likely to prefer stronger models. Here’s a visualization of predicted human preference for an easy prompt (“hello, how are you?”) and a challenging prompt (“Explain why Planc length …”).

Preference predictors make it possible to create leaderboards unique to any prompt and domain.

clem Nganx andrewrreed vladbogo osanseviero loubnabnl EmilyWitko ankity09 samusenps callmesan bedestaz ChavyvAkvar dortorwu2 gsarti Shaheer-ipynb aotrih ntdas dark-pen julien-c kgourgou pcuenq valeriiakuka sbrandeis

andrewrreed mvaloatto kramp julien-c kgourgou gigant Bkarine


👷🏽♀📚🔨  Announcing the Foundation Model Development Cheatsheet!

My first 🤗Post🤗 ever to announce the release of a fantastic collaborative resource to support model developers across the full development stack: The FM Development Cheatsheet available here:  https://fmcheatsheet.org/

The cheatsheet is a growing database of the many crucial resources coming from open research and development efforts to support the responsible development of models. This new resource highlights essential yet often underutilized tools in order to make it as easy as possible for developers to adopt best practices, covering among other aspects:
🧑🏼🤝🧑🏼 data selection, curation, and governance;
📖 accurate and limitations-aware documentation;
⚡ energy efficiency throughout the training phase;
📊 thorough capability assessments and risk evaluations;
🌏 environmentally and socially conscious deployment strategies.

We strongly encourage developers working on creating and improving models to make full use of the tools listed here, and to help keep the resource up to date by adding the resources that you yourself have developed or found useful in your own practice 🤗

Congrats to all the participants in this effort for the release! Read more about it from:
@Shayne - https://twitter.com/ShayneRedford/status/1763215814860186005
@hails and @stellaathena - https://blog.eleuther.ai/fm-dev-cheatsheet/
@alon-albalak - http://nlp.cs.ucsb.edu/blog/a-new-guide-for-the-responsible-development-of-foundation-models.html

And also to  @gabrielilharco @sayashk @kklyman @kylel @mbrauh @fauxneticien @avi-skowron @Bertievidgen Laura Weidinger, Arvind Narayanan, @VictorSanh @Davlan @percyliang Rishi Bommasani, @breakend @sasha 🔥

VictorSanh hails mohammedbriman Shayne kklyman pietrolesci osanseviero sayhan poliva loubnabnl yongzx evdcush alon-albalak boapps mbrauh Epiculous BrigitteTousi mathiasn1 sasha de-Rodrigo alielfilali01

hails mohammedbriman Shayne kklyman mvaloatto osanseviero evdcush alon-albalak Epiculous BrigitteTousi


Defending LLMs against Jailbreaking Attacks via Backtranslation (2402.16459)

https://huggingface.co/papers/2402.16459
**Defending LLMs against Jailbreaking Attacks via Backtranslation**

I really love this! Its a really innovative way to get robust defense against jailbreaking. Its not cheap, 2-3 calls per user request. But for some use-cases it can be worth it!


8 Spaces Of The Week is nice, but 840 is even better! 🔥

Here is the complete library of ALL Spaces featured by Hugging Face since October 2021:

All Spaces Of The Week - https://huggingface.co/spaces/mvaloatto/ASOTW

-
A special mention goes to @osanseviero, whose collection inspired me to design this dedicated Space. Another shoutout to @victor, whose intricately designed Spaces cards motivated me to step up my CSS game :) I plan to release additional features in the future. In the meantime, suggestions are welcome!

osanseviero vladbogo hysts victor kramp clem samusenps fffiloni nbroad Nymbo


An increasing number of engineers and researchers are developing foundational models. Navigating the tools, resources, codebases, and best practices guides is daunting for new contributors.

Introducing the Foundation Model Development Cheatsheet, a succinct guide with 250+ resources & tools for:
📖 sourcing data
🔍 documenting & audits
🌍 environmental impact
🥊 risks & harms eval
🎮 release & monitoring

https://fmcheatsheet.org/

👐 What tools & resources should appear in that cheatsheet? Contributions encouraged!

This is the result of a large collaboration between many organizations promoting open-science, and spearheaded by @Shayne 🔥

Shayne osanseviero kirch mayacinka mvaloatto ajibawa-2023 FranzNo yjernite erikab samusenps dkyazze sbarman25 victor lixinjie


Training a SOTA code LLM with a fully transparent library (nanotron) built from scratch -> Done ✅

⭐  BlogPost: https://huggingface.co/blog/starcoder2
🔗 StarCoder2 collection: https://huggingface.co/collections/bigcode/starcoder2-65de6da6e87db3383572be1a


🙋🏻♂ hey there folks ,

Star coder came out and it's really fascinating in more ways than one !

first off it codes well already. but secondly it's reported to "know" 101 programming languages !

that actually means it's ripe for fine tunes, so if you're like me you've been bookmarking cool datasets and cant wait to get started !

that said , here's a cool demo where you can try it out now : https://huggingface.co/spaces/Tonic/starcoder2

turns out it can program a T5 demo using gradio !

victor osanseviero kgourgou clem fffiloni

mvaloatto ajibawa-2023 clem goncharenko yxxsgdmn


Where I work, we are obsessed with what happens to a model's performance after it has been deployed. We call this post-deployment data science.

Let me tell you about a post-deployment data science algorithm that we recently developed to measure the impact of Concept Drift on a model's performance.

How can we detect Concept Drift? 🤔

All ML models are designed to do one thing: learning a probability distribution in the form of P(y|X). In other words, they try to learn how to model an outcome 'y' given the input variables 'X'. 🧠

This probability distribution, P(y|X), is also called Concept. Therefore, if the Concept changes, the model may become invalid.

❓But how do we know if there is a new Concept in our data?
❓Or, more important, how do we measure if the new Concept is affecting the model's performance?

💡 We came up with a clever solution where the main ingredients are a reference dataset, one where the model's performance is known, and a dataset with the latest data we would like to monitor.

👣 Step-by-Step solution:

1⃣ We start by training an internal model on a chunk of the latest data. ➡ This allows us to learn the new possible Concept presented in the data.

2⃣ Next, we use the internal model to make predictions on the reference dataset.

3⃣ We then estimate the model's performance on the reference dataset, assuming the model's predictions on the monitoring data as ground truth.

4⃣ If the estimated performance of the internal model and the actual monitored model are very different, we then say that there has been a Concept Drift.

To quantify how this Concept impacts performance, we subtract the actual model's performance on reference from the estimated performance and report a delta of the performance metric. ➡ This is what the plot below shows. The change of the F1-score due to Concept drift! 🚨

This process is repeated for every new chunk of data that we get. 🔁

ajibawa-2023 fblgit jessicagab Nhebo victor vishwask codito anujd9


Real-time object detection w/ 🤗 Transformers.js, running YOLOv9 locally in your browser! 🤯

Try it out yourself: https://huggingface.co/spaces/Xenova/video-object-detection
(Model used + example code: https://huggingface.co/Xenova/gelan-c_all)

This demo shows why on-device ML is so important:
1. Privacy - local inference means no user data is sent to the cloud
2. No server latency - empowers developers to build real-time applications
3. Lower costs - no need to pay for bandwidth and processing of streamed video

I can't wait to see what you build with it! 🔥

DmitryRyumin macadeliccc giux78 radames ajibawa-2023 osanseviero Dlbk jaickerag victor mvaloatto clem samusenps bwang0911 Thepickledegg hogunkim felixdrp Noomam Youngwon elcrei


I have dedicated several days, working over 12 hours each day, on SUPIR (Scaling-UP Image Restoration), a cutting-edge image enhancement and upscaling model introduced in the paper Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild.

This model is simply mind-blowing. At the bottom of this post, you will see side-by-side comparisons of SUPIR versus the extremely expensive online service, Magnific AI. Magnific is known to be the best among the community. However, SUPIR is by far superior. SUPIR also significantly outperforms Topaz AI upscale. SUPIR manages to remain faithful to the original image almost 100% while adding details and achieving super upscaling with the best realism.

You can read the full blog post here : https://huggingface.co/blog/MonsterMMORPG/supir-sota-image-upscale-better-than-magnific-ai

ameerazam08 clem iDrops samusenps diogofranciscop yxxsgdmn Johnnycadelover


🔥 What's that biomedical model that got 170,763 downloads last month on HuggingFace?! Well, the paper is finally published! #BioLORD

📰 Read our article in the Journal of the American Medical Informatics Association:
https://academic.oup.com/jamia/advance-article/doi/10.1093/jamia/ocae029/7614965

📝`TLDR:` BioLORD-2023 is a series of semantic language models for the biomedical domain, capable of representing clinical concepts and sentences in a semantic space aligned with human preferences. Our new multilingual version supports 50+ languages and is further finetuned on 7 European languages. These models were trained contrastively and through distillations, using a corpus unifying in the same latent space the concept names of biomedical concepts and their descriptions. For concepts which didn't have a description written by humans in UMLS, we use information contained in the SnomedCT knowledge graph and the capabilities of ChatGPT to generate synthetic data and improve our results.

🤗 Access our models from the HuggingFace hub, including the new 2023-C and 2023-S variants:
https://huggingface.co/FremyCompany/BioLORD-2023
https://huggingface.co/FremyCompany/BioLORD-2023-M
https://huggingface.co/FremyCompany/BioLORD-2023-S
https://huggingface.co/FremyCompany/BioLORD-2023-C

ilGawo cdevelder osanseviero katielink julien-c clem macadeliccc UMCU danielhanchen samusenps yxxsgdmn pdelobelle


The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits (2402.17764)

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

https://huggingface.co/papers/2402.17764

Recent research, such as BitNet, is paving the way for a new era of 1-bit Large Language Models (LLMs). In this work, we introduce a 1-bit LLM variant, namely BitNet b1.58, in which every single parameter (or weight) of the LLM is ternary {-1, 0, 1}. It matches the full-precision (i.e., FP16 or BF16) Transformer LLM with the same model size and training tokens in terms of both perplexity and end-task performance, while being significantly more cost-effective in terms of latency, memory, throughput, and energy consumption. More profoundly, the 1.58-bit LLM defines a new scaling law and recipe for training new generations of LLMs that are both high-performance and cost-effective. Furthermore, it enables a new computation paradigm and opens the door for designing specific hardware optimized for 1-bit LLMs.

Azamat1k bofenghuang osanseviero clem macadeliccc ssone95 minjejeon SRDdev arcdyn ajibawa-2023 Kukedlc yongzx eramax mvaloatto kgourgou danielhanchen arjunsriva goncharenko notune koflerdavid krishnapraveen Banshal Hemanth-thunder Aishou northern-64bit tuanlda78202 andrewrreed CarlLee rippertnt joecrypto feveromo mathiasn1 RachidAR SaiNikhileshReddy yxxsgdmn julien-c femboysLover victor wath5

clem macadeliccc giux78 ssone95 fblgit adhisetiawan tuanlda78202 samusenps mindrage Parth SaiNikhileshReddy julien-c ncard


⭐ Today we’re releasing The Stack v2 & StarCoder2:  a series of 3B, 7B & 15B code generation models trained on 3.3 to 4.5 trillion tokens of code:

- StarCoder2-15B matches or outperforms CodeLlama 34B, and approaches DeepSeek-33B on multiple benchmarks.
- StarCoder2-3B outperforms StarCoderBase-15B and similar sized models.
- The Stack v2 a 4x larger dataset than the Stack v1, resulting in 900B unique code tokens 🚀
As always, we released everything from models and datasets to curation code. Enjoy!

🔗  StarCoder2 collection: https://huggingface.co/collections/bigcode/starcoder2-65de6da6e87db3383572be1a
🔗  Paper: https://drive.google.com/file/d/17iGn3c-sYNiLyRSY-A85QOzgzGnGiVI3/view
🔗  BlogPost: https://huggingface.co/blog/starcoder2
🔗  Code Leaderboard: https://huggingface.co/spaces/bigcode/bigcode-models-leaderboard

lvwerra DmitryRyumin thomwolf olivierdehaene smangrul osanseviero mcpotato alielfilali01 SivilTaram victor yuxiang630 BrigitteTousi VictorSanh harmdevries RaymondLi 3outeille Azamat1k vladbogo bofenghuang ncoop57 philschmid Avremi clem admarcosai macadeliccc mayank-mishra hysts kramp nouamanetazi danielhanchen goncharenko MoritzLaurer tbeck not-lain amyeroberts andrewrreed yxxsgdmn seyf1elislam omaryshchenko mishig

lvwerra thomwolf olivierdehaene smangrul osanseviero mcpotato alielfilali01 SivilTaram victor yuxiang630 BrigitteTousi VictorSanh harmdevries RaymondLi 3outeille ncoop57 philschmid clem admarcosai macadeliccc EdoAbati euclaise ajibawa-2023 mayank-mishra dalraf nouamanetazi tbeck not-lain amyeroberts andrewrreed samusenps seyf1elislam arjunguha mishig Ramikan-BR

lvwerra thomwolf olivierdehaene smangrul osanseviero SivilTaram victor yuxiang630 BrigitteTousi RaymondLi 3outeille hlarcher OmBenz philschmid Abderrazak clem admarcosai mayank-mishra not-lain mishig


EMO: Emote Portrait Alive - Generating Expressive Portrait Videos with
  Audio2Video Diffusion Model under Weak Conditions (2402.17485)

🌟🎭✨ Exciting News! The Latest in Expressive Video Portrait Generation! 🌟🎭✨

📄 Title: EMO: Emote Portrait Alive - Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions

👥 Authors: Linrui Tian, @lucaskingjade, Bang Zhang, and @Liefeng

🔗 Paper: https://huggingface.co/papers/2402.17485
🔗 Github Page: https://humanaigc.github.io/emote-portrait-alive
🔗 Repository: https://github.com/HumanAIGC/EMO

🔍 Keywords: #EMO #EmotePortrait #Audio2VideoDiffusion #ExpressiveAnimations #VideoGeneration  #DigitalArt #HumanExpression #ComputerVision #DeepLearning #AI

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

s3nh osanseviero DmitryRyumin Azamat1k aloobun sachaarbonel plmsmile ajibawa-2023 victor mvaloatto Hemanth-thunder samusenps CedrickChu thomwolf newxuyangcao oviniciusfeitosa


Super excited to share with you all our latest contribution from 2A2I.

Today we announce : https://huggingface.co/datasets/2A2I/Arabic-OpenHermes-2.5

Arabic-OpenHermes-2.5 Is simply the translation of the original dataset released by @teknium couple months ago !
In fact it looks as a simple task ! In reality it was quite a laborious job !
But thanks to @maghwa & @medmac01 this dataset managed to see the light today and help creating better  / more aligned arabic LLMs in the near future.

If you are interested to join us and/or help us, please leave a comment below or visit our HuggingFace Org Card for more details about How/What you can do.

More datasets to come and more models are in the way 🔥

medmac01 NovoCode osanseviero ajibawa-2023 maghwa mvaloatto sayhan samusenps


🚨 New Release of 🤗PEFT!

1. New methods for merging LoRA weights. Refer this HF Post for more details: https://huggingface.co/posts/smangrul/850816632583824

2. AWQ and AQLM support for LoRA. You can now:
- Train adapters on top of 2-bit quantized models with AQLM
- Train adapters on top of powerful AWQ quantized models
Note for inference you can't merge the LoRA weights into the base model!

3. DoRA support: Enabling DoRA is as easy as adding `use_dora=True` to your `LoraConfig`. Find out more about this method here: https://arxiv.org/abs/2402.09353

4. Improved documentation, particularly docs regarding PEFT LoRA+DeepSpeed and PEFT LoRA+FSDP! 📄 Check out the docs at https://huggingface.co/docs/peft/index.

5. Full Release Notes: https://github.com/huggingface/peft/releases/tag/v0.9.0

osanseviero Kukedlc clem Ji-Ha samusenps alielfilali01 mudogruer ybelkada ryokeken Cstark dahwinsingularity

NickyNicky Jenish-23 ixaxaar ababio rizwan-ai ybelkada Cstark ZennyKenny catastropiyush


🆕 There's now a visible description on Spaces previews. You can set the description of your Spaces by editing their README.md - it should make your Spaces more discoverable 🚀.

Felladrin osanseviero mvaloatto clefourrier alielfilali01 clem Vokturz samusenps ojasvisingh786


CausalGym: Benchmarking causal interpretability methods on linguistic
  tasks (2402.12560)

🔍 Today's pick in Interpretability & Analysis of LMs: CausalGym: Benchmarking causal interpretability methods on linguistic tasks by @aryaman D. Jurafsky @cgpotts

TL;DR: Introduce a revisited benchmark to evaluate the effectiveness and reliability of intervention methods across several linguistic phenomena.

While several interpretability methods are currently used to discover task-relevant model components, their performance and reliability is seldom tested broadly.

This paper adapts the SyntaxGym benchmark, originally conceived for the study of psycholinguistic phenomena such as subject-verb agreement and garden-path sentences, to evaluate intervention-based interpretability methods. In practice, faithful interventions over model components are expected to cause a predictable change in model prediction (e.g. singular -> plural verb).

Various methods are benchmarked on Pythia models ranging from 14M to 6.9B params, finding Distributed Alignment Search (DAS) to consistently outperform other approaches, followed by probing. When recurring to control tasks to account for the expressivity of supervised methods, probing is found to be more reliable than DAS in larger model sizes.

Authors conclude with an evaluation of how features driving linguistically plausible behaviours emerge during model training. These features are observed to emerge in Pythia models after 1k training steps, and become progressively more complex over time.

📄 Paper: https://huggingface.co/papers/2402.12560
💻 Code: https://github.com/aryamanarora/causalgym
🔡 Dataset: https://huggingface.co/datasets/aryaman/causalgym

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9


ChatMusician: Understanding and Generating Music Intrinsically with LLM (2402.16153)

ChatMusician

Understanding and Generating Music Intrinsically with LLM

https://huggingface.co/papers/2402.16153

While Large Language Models (LLMs) demonstrate impressive capabilities in text generation, we find that their ability has yet to be generalized to music, humanity's creative language. We introduce ChatMusician, an open-source LLM that integrates intrinsic musical abilities. It is based on continual pre-training and finetuning LLaMA2 on a text-compatible music representation, ABC notation, and the music is treated as a second language. ChatMusician can understand and generate music with a pure text tokenizer without any external multi-modal neural structures or tokenizers. Interestingly, endowing musical abilities does not harm language abilities, even achieving a slightly higher MMLU score. Our model is capable of composing well-structured, full-length music, conditioned on texts, chords, melodies, motifs, musical forms, etc, surpassing GPT-4 baseline. On our meticulously curated college-level music understanding benchmark, MusicTheoryBench, ChatMusician surpasses LLaMA2 and GPT-3.5 on zero-shot setting by a noticeable margin. Our work reveals that LLMs can be an excellent compressor for music, but there remains significant territory to be conquered. We release our 4B token music-language corpora MusicPile, the collected MusicTheoryBench, code, model and demo in GitHub.

regikono euclaise mexicanamerican ajibawa-2023 mvaloatto danielhanchen korhancagla osanseviero kramp victor Azamat1k Ranjanj1 Hoctar77 rafaelpierrehf

clem danielhanchen osanseviero EquinoxElahin asus4


Gemma QLoRA finetuning is now 2.4x faster and uses 58% less VRAM than FA2 through 🦥Unsloth! Had to rewrite our Cross Entropy Loss kernels to work on all vocab sizes, re-design our manual autograd engine to accept all activation functions, and more! I wrote all about our learnings in our blog post: https://unsloth.ai/blog/gemma.

Also have a Colab notebook with no OOMs, and has 2x faster inference for Gemma & how to merge and save to llama.cpp GGUF & vLLM: https://colab.research.google.com/drive/10NbwlsRChbma1v55m8LAPYG15uQv6HLo?usp=sharing

And uploaded 4bit pre-quantized versions for Gemma 2b and 7b: https://huggingface.co/unsloth/gemma-7b-bnb-4bit https://huggingface.co/unsloth/gemma-2b-bnb-4bit

```
from unsloth import FastLanguageModel
model, tokenzer = FastLanguageModel.from_pretrained("unsloth/gemma-7b")
model = FastLanguageModel.get_peft_model(model)
```

Sylvestre harpreetsahota cnmoro clem osanseviero ybelkada rbiswasfc alielfilali01 danielus ambivalent02 victor


The open-source AI community can build impactful datasets collectively!

Announcing https://huggingface.co/datasets/DIBT/10k_prompts_ranked, the first dataset release from Data Is Better Together.

Created in <2 weeks by the community. Includes:

✨ 10,000+ prompt quality ratings
🧑💻 Human and synthetic data prompts
🌐 Generated by 300+ contributors

How and why collaborative datasets?

It's no secret that high-quality open data is essential for creating better open models. The open source community shares 100s of models, datasets and demos openly weekly, but collectively building open datasets has been less explored.

Datasets have a massive role in shaping what models can be created. If we want more high-quality models for all languages, domains and tasks, we need more and better open datasets for all languages, domains and tasks!

To explore how the community could build impactful datasets collectively, Argilla added support for HF authentication for Argilla instances hosted on a Hugging Face Space. Anyone with an HF login could begin contributing to a dataset in <1 minute.

To test this new workflow, we launched a task to rank the quality of prompts (human and synthetically generated).

In less than two weeks, we built a community of over 300 contributors for this dataset 🤗

This dataset became a reality thanks to the dedication of all the individuals who lent their support ❤ To see the amazing people behind this dataset, visit https://huggingface.co/spaces/DIBT/prompt-collective-dashboard

This is just the start for collectively building powerful open datasets!

osanseviero alvarobartt dvilasuero samusenps clem TuringsSolutions danielhanchen alielfilali01 mmhamdy Tonic

victor osanseviero alvarobartt dvilasuero DmitryRyumin Sylvestre clem danielhanchen alielfilali01


🚀 Just released version 0.21.0 of the `huggingface_hub` Python library!

Exciting updates include:
🖇 Dataclasses everywhere for improved developer experience!
💾 HfFileSystem optimizations!
🧩 `PyTorchHubMixin` now supports configs and safetensors!
✨ `audio-to-audio` supported in the InferenceClient!
📚 Translated docs in Simplified Chinese and French!
💔 Breaking changes: simplified API for listing models and datasets!

Check out the full release notes for more details: https://huggingface.co/spaces/Wauplin/huggingface_hub/discussions/4 🤖💻

DmitryRyumin bmuskalla pcuenq osanseviero lunarflu clem severo pierrci harpreetsahota digiplay samusenps ajibawa-2023 dillfrescott MoritzLaurer dalraf victor mvaloatto yjernite danielus Kvikontent tonyassi radames

alvarobartt osanseviero lunarflu clem pierrci harpreetsahota dillfrescott yjernite radames

fffiloni kramp harpreetsahota dillfrescott radames