next version of https://huggingface.co/datasets/sequelbox/Celestia will be https://huggingface.co/datasets/microsoft/orca-agentinstruct-1M-v1 style. coming soon


Minimalistic Adapters 🎃

🚀Demo Here:
 https://huggingface.co/spaces/prithivMLmods/FLUX-LoRA-DLC

🚀Model:
{ Quote Tuner } : https://huggingface.co/prithivMLmods/Flux.1-Dev-Quote-LoRA
{ Stamp Art } : https://huggingface.co/prithivMLmods/Flux.1-Dev-Stamp-Art-LoRA
{ Hand Sticky } : https://huggingface.co/prithivMLmods/Flux.1-Dev-Hand-Sticky-LoRA
{ Poster HQ } : https://huggingface.co/prithivMLmods/Flux.1-Dev-Poster-HQ-LoRA
{ Ctoon Min } : https://huggingface.co/prithivMLmods/Flux.1-Dev-Ctoon-LoRA

🚀Collection:
{ Flux LoRA Collection} : https://huggingface.co/collections/prithivMLmods/flux-lora-collections-66dd5908be2206cfaa8519be
{ LoRA Space Collection } : https://huggingface.co/collections/prithivMLmods/lora-space-collections-6714b72e0d49e1c97fbd6a32

🚀For More Visit
https://huggingface.co/strangerzonehf
.
.
.
🤗@prithivMLmods

Ngrthm RenderIo darksfx ai4life44 hypergod rdrede


Kohya brought massive improvements to FLUX LoRA (as low as 4 GB GPUs) and DreamBooth / Fine-Tuning (as low as 6 GB GPUs) training - check attached images in full size to see full details

You can download all configs and full instructions

> https://www.patreon.com/posts/112099700 - Fine Tuning post

> https://www.patreon.com/posts/110879657 - LoRA post

Kohya brought massive improvements to FLUX LoRA and DreamBooth / Fine-Tuning (min 6GB GPU) training.

Now as low as 4GB GPUs can train FLUX LoRA with decent quality and 24GB and below GPUs got a huge speed boost when doing Full DreamBooth / Fine-Tuning training

You need minimum 4GB GPU to do a FLUX LoRA training and minimum 6 GB GPU to do FLUX DreamBooth / Full Fine-Tuning training. It is just mind blowing.

You can download all configs and full instructions > https://www.patreon.com/posts/112099700

The above post also has 1-click installers and downloaders for Windows, RunPod and Massed Compute

The model downloader scripts also updated and downloading 30+GB models takes total 1 minute on Massed Compute

You can read the recent updates here : https://github.com/kohya-ss/sd-scripts/tree/sd3?tab=readme-ov-file#recent-updates

This is the Kohya GUI branch : https://github.com/bmaltais/kohya_ss/tree/sd3-flux.1

Key thing to reduce VRAM usage is using block swap

Kohya implemented the logic of OneTrainer to improve block swapping speed significantly and now it is supported for LoRAs as well

Now you can do FP16 training with LoRAs on 24 GB and below GPUs

Now you can train a FLUX LoRA on a 4 GB GPU - key is FP8, block swap and using certain layers training (remember single layer LoRA training)

It took me more than 1 day to test all newer configs, their VRAM demands, their relative step speeds and prepare the configs :)


Ok RNNs can rap too:)

Here we implement the seminal RNN paper “Generating Text with Recurrent Neural Networks"- we train a character-level multiplicative recurrent neural network model (~250k params) for 1000 epochs with Adam opt on 2pac's "Hit 'em Up", sample was fun lol.

Code: https://github.com/Jaykef/ai-algorithms/blob/main/generating_texts_with_rnns.ipynb


Finaly I realesed mediapipe-face animation space.

Mediapipe 68-points Eyes-Closed and Mouth-Opened
https://huggingface.co/spaces/Akjava/mediapipe-68-facial-guide-eyes-closed-mouth-opened

[Article]Results: Converted Guide Images(eyes-closed and mouth-opened) with Flux.1 schenll img2img/inpaint
https://huggingface.co/blog/Akjava/result-guide-image-eyes-mouth

All the other tools listed are designed to support Mediapipe Face Animation

https://huggingface.co/collections/Akjava/mediapipe-tools-672ffe8ee7b62763c31b70c7

https://huggingface.co/collections/Akjava/webp-3-frame-talking-animation-tools-672819ce4989f354cdbcc739


Good folks at @nvidia  and @Tsinghua_Uni have released LLAMA-MESH - A Revolutionary Approach to 3D Content Generation!

This innovative framework enables the direct generation of 3D meshes from natural language prompts while maintaining strong language capabilities.

Here is the Architecture & Implementation!

>> Core Components

Model Foundation
- If you haven't guessed it yet, it's built on the LLaMA-3.1-8B-Instruct base model
- Maintains original language capabilities while adding 3D generation
- Context length is set to 8,000 tokens

3D Representation Strategy
- Uses the OBJ file format for mesh representation
- Quantizes vertex coordinates into 64 discrete bins per axis
- Sorts vertices by z-y-x coordinates, from lowest to highest
- Sorts faces by the lowest vertex indices for consistency

Data Processing Pipeline
- Filters meshes to a maximum of 500 faces for computational efficiency
- Applies random rotations (0°, 90°, 180°, 270°) for data augmentation
- Generates ~125k mesh variations from 31k base meshes
- Uses Cap3D-generated captions for text descriptions

>> Training Framework

Dataset Composition
- 40% Mesh Generation tasks
- 20% Mesh Understanding tasks
- 40% General Conversation (UltraChat dataset)
- 8x training turns for generation, 4x for understanding

Training Configuration
- Deployed on 32 A100 GPUs (for Nvidia, this is literally in-house)
- 21,000 training iterations
- Global batch size: 128
- AdamW optimizer with a 1e-5 learning rate
- 30-step warmup with cosine scheduling
- Total training time: approximately 3 days (based on the paper)

This research opens exciting possibilities for intuitive 3D content creation through natural language interaction. The future of digital design is conversational!


OmniVision-968M: a new local VLM for edge devices, fast & small but performant
💨 a new vision language model with 9x less image tokens, super efficient
📖 aligned with DPO for reducing hallucinations
⚡ Apache 2.0 license 🔥

Demo hf.co/spaces/NexaAIDev/omnivlm-dpo-demo
Model https://huggingface.co/NexaAIDev/omnivision-968M


Kokoro: a small, fast 80M param TTS model hosted on ZeroGPU at https://hf.co/spaces/hexgrad/Kokoro-TTS


I've built a small open utility pip package called LLM-Forwarder that allows you to inject context, such as adding a private RAG, into existing chat applications by forwarding the app through the LLM-Forwarder. In the forwarder server, you can configure custom code to re-process chat messages and alter the user prompt, for example, by adding extra context.

https://pypi.org/project/llm-forwarder/
More details
https://devquasar.com/llmforwarder/


Mann-E's new platform is up and running.

You can access our platform here at https://mann-e.com. We're still working on it and reducing the bugs and we also are trying to add a guest session which lets you make images as guests.

What do you think?


It's not every day you see the No. 1 ranked paper of the day open-sourcing a very powerful image editing app!

Fascinating to see MagicQuill - a groundbreaking interactive image editing system that makes precise photo editing effortless through advanced AI!

The system's architecture features three sophisticated components:

1. Editing Processor:
- Implements a dual-branch architecture integrated into a latent diffusion framework
- Utilizes PiDiNet for edge map extraction and content-aware per-pixel inpainting
- Features a specialized UNet architecture with zero-convolution layers for feature insertion
- Employs denoising score matching for training the control branch
- Processes both structural modifications via scribble guidance and color manipulation through downsampled color blocks
- Maintains pixel-level control through VAE-based latent space operations

2. Painting Assistor:
- Powered by a fine-tuned LLaVA multimodal LLM using Low-Rank Adaptation (LoRA)
- Trained on a custom dataset derived from Densely Captioned Images (DCI)
- Processes user brushstrokes through specialized Q&A tasks for add/subtract/color operations
- Features bounding box coordinate normalization for precise stroke localization
- Implements streamlined single-word/phrase outputs for real-time performance

3. Idea Collector:
- Built as a modular ReactJS component library
- Supports cross-platform deployment via HTTP protocols
- Compatible with Gradio and ComfyUI frameworks
- Features comprehensive layer management and parameter adjustment capabilities
- Implements real-time canvas updates and preview generation

The system outperforms existing solutions like SmartEdit and BrushNet in edge alignment and color fidelity while maintaining seamless integration with popular AI frameworks.

What are your thoughts on AI-powered creative tools?


QT your 🕊Hope🕊 and ⚖Justice⚖ art🎨

🚲 Stolen bike in Denver FOUND!
 - Sometimes hope & justice DO prevail!

🎬 Created an AI+Art+Music tribute:
 -🧠 AI App that Evaluates GPT-4o vs Claude: https://t.co/odrYdaeizZ
 #GPT #Claude #Huggingface @OpenAI @AnthropicAI pic.twitter.com/Q9wGNzLm5C
— Aaron Wacker (@Aaron_Wacker) November 16, 2024

🕊Hope🕊 and ⚖Justice⚖ AI
🚲 Stolen bike in Denver FOUND - Sometimes hope & justice DO prevail.

🎬 So I Created an AI+Art+Music tribute:
   -🧠 AI App that Evaluates GPT-4o vs Claude:
https://huggingface.co/spaces/awacke1/RescuerOfStolenBikes
https://x.com/Aaron_Wacker/status/1857640877986033980?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1857640877986033980%7Ctwgr%5E203a5022b0eb4c41ee8c1dd9f158330216ac5be1%7Ctwcon%5Es1_c10&ref_url=https%3A%2F%2Fpublish.twitter.com%2F%3Furl%3Dhttps%3A%2F%2Ftwitter.com%2FAaron_Wacker%2Fstatus%2F1857640877986033980

```html
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">QT your 🕊Hope🕊 and ⚖Justice⚖ art🎨<br><br>🚲 Stolen bike in Denver FOUND! <br> - Sometimes hope & justice DO prevail! <br><br>🎬 Created an AI+Art+Music tribute: <br> -🧠 AI App that Evaluates GPT-4o vs Claude: <a href="https://t.co/odrYdaeizZ">https://t.co/odrYdaeizZ</a><br> <a href="https://twitter.com/hashtag/GPT?src=hash&ref_src=twsrc%5Etfw">#GPT</a> <a href="https://twitter.com/hashtag/Claude?src=hash&ref_src=twsrc%5Etfw">#Claude</a> <a href="https://twitter.com/hashtag/Huggingface?src=hash&ref_src=twsrc%5Etfw">#Huggingface</a> <a href="https://twitter.com/OpenAI?ref_src=twsrc%5Etfw">@OpenAI</a> <a href="https://twitter.com/AnthropicAI?ref_src=twsrc%5Etfw">@AnthropicAI</a> <a href="https://t.co/Q9wGNzLm5C">pic.twitter.com/Q9wGNzLm5C</a></p>— Aaron Wacker (@Aaron_Wacker) <a href="https://twitter.com/Aaron_Wacker/status/1857640877986033980?ref_src=twsrc%5Etfw">November 16, 2024</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
```

#GPT #Claude #Huggingface
@OpenAI
@AnthropicAI


What if I told you that LLM models do not simply predict the next token in a sequence but instead utilize an emergent structural pattern-based system to comprehend language and concepts? I created a graph-based optimizer that not only works, but it also actually beats Adam, like very badly. I prove it thoroughly using SMOL LLM models. The secret? The graph is not what you think it is, humans. Code, full explanation, and more in this video. The Rhizome Optimizer is MIT licensed.  I have completed my research. I fully understand now.

https://youtu.be/OMCRRueMhdI


We’re excited to release Abstract2Appendix v1 10K , a high-quality dataset crafted to enhance the long-context capabilities of Large Language Models (LLMs). This dataset combines thousands of peer reviews from NeurIPS 2023, EMNLP 2023, TMLR, and ICLR 2023, making it a treasure trove of detailed feedback, critical reasoning, and structured academic insights. Our experiments showed that this dataset increased long context ability of phi-3 models!

🌟 Key Highlights:

	•	Expert Reviews: Aggregated from 3–6 reviews per paper, capturing the most insightful and constructive content.
	•	Rich Metadata: we have aggregated the reviews, and also included full parsed paper
	•	LLM Ready: Perfect for fine-tuning (We did dpo and sft)

🎯 Use Cases:

	•	Fine-tuning models with Direct Preference Optimization (DPO) and Supervised Fine-Tuning (SFT).
	•	Benchmarking zero-shot and long-context comprehension capabilities.

🔗 Explore the dataset: https://huggingface.co/datasets/alexshengzhili/Abstract2Appendix_v1_10k

This dataset is based on the methodology described in our recent paper, “Abstract2Appendix: Academic Reviews Enhance LLM Long-Context Capabilities”. Check it out for more details! https://arxiv.org/abs/2411.05232


📢 For those who are interested in extracting information about ✍ authors from texts, happy to share personal 📹  on Reading Between the lines: adapting ChatGPT-related systems 🤖  for Implicit Information Retrieval National

Youtube: https://youtu.be/nXClX7EDYbE

🔑 In this talk, we refer to IIR as such information that is indirectly expressed by ✍ author / 👨 character / patient / any other entity.

📊  I cover the 1⃣  pre-processing and 2⃣  reasoning techniques, aimed at enhancing gen AI capabilities in IIR. To showcase the effectiveness of the proposed techniques, we experiment with such IIR tasks as Sentiment Analysis, Emotion Extraction / Causes Prediction.

In pictures below, sharing the quick takeaways on the pipeline construction and experiment results 🧪

Related paper cards:
📜 emotion-extraction: https://nicolay-r.github.io/#semeval2024-nicolay
📜 sentiment-analysis: https://nicolay-r.github.io/#ljom2024

Models:
https://huggingface.co/nicolay-r/flan-t5-tsa-thor-base
https://huggingface.co/nicolay-r/flan-t5-emotion-cause-thor-base

📓 PS: I got a hoppy for advetising HPMoR  ✨ 😁


Hello Hugging Face Community,

I'd like to share here a bit more about our Deep Learning Containers (DLCs) we built with Google Cloud, to transform the way you build AI with open models on this platform!

With pre-configured, optimized environments for PyTorch Training (GPU) and Inference (CPU/GPU), Text Generation Inference (GPU), and Text Embeddings Inference (CPU/GPU), the Hugging Face DLCs offer:

⚡ Optimized performance on Google Cloud's infrastructure, with TGI, TEI, and PyTorch acceleration.
🛠 Hassle-free environment setup, no more dependency issues.
🔄 Seamless updates to the latest stable versions.
💼 Streamlined workflow, reducing dev and maintenance overheads.
🔒 Robust security features of Google Cloud.
☁ Fine-tuned for optimal performance, integrated with GKE and Vertex AI.
📦 Community examples for easy experimentation and implementation.
🔜 TPU support for PyTorch Training/Inference and Text Generation Inference is coming soon!

Find the documentation at https://huggingface.co/docs/google-cloud/en/index
If you need support, open a conversation on the forum: https://discuss.huggingface.co/c/google-cloud/69

odin0 John6666 jcudit prithivMLmods qftop Sri-Vigneshwar-DJ


Hello Hugging Face community!

I wanted to introduce myself and my company @Overlaiapp. We are a collective of filmmakers, photographers, and AI engineers working on high resolution (8K+) training data.

We plan to share a lot of our datasets with the community and are kicking things off with two curated datasets:

- https://huggingface.co/datasets/Overlaiai/OregonCoastin4K

- https://huggingface.co/datasets/Overlaiai/SubArcticPolarBear

Overlai.ai Dataset Features

🎥 Oversampled: Every clip is captured in stunning 8K resolution, delivering rich detail ideal for fine tuning scenic landscapes and ocean dynamics.

📸 Variance: Includes close-up details, slow-motion footage of crashing waves, sweeping landscapes, and wildlife shots.

📋 Detailed Metadata: Every clip is paired with structured metadata, including creative descriptions, precise camera movements, lens information, field of view calculations, and shot settings, ensuring AI models can fully understand and replicate real-world cinematography with accuracy.

⚙ Consistency: Re-thinking training data at the point of capture by "overshooting" a subject, enabling models to learn more nuanced relationships and views across scenes.

🌅 Light: Shot during early morning and sunset light for optimal color contrast and dynamic range, maximizing visual quality for color and lighting-sensitive tasks.

🔍 Curation: Curated specifically for machine learning, providing clean, high-quality data for next generation model training.

prithivMLmods csabakecskemeti jeffcookio mmhamdy


Animaker / Anime Engine : 🤗

Only on🚀: https://huggingface.co/strangerzonehf

Demo: https://huggingface.co/spaces/prithivMLmods/FLUX-LoRA-DLC

Adapters:
->Model 1: https://huggingface.co/strangerzonehf/Flux-Animeo-v1-LoRA
->Model 2: https://huggingface.co/strangerzonehf/Flux-Animex-v2-LoRA

Collections:
->Flux LoRA Collection: https://huggingface.co/collections/prithivMLmods/flux-lora-collections-66dd5908be2206cfaa8519be
->Stranger Zones: https://huggingface.co/collections/prithivMLmods/stranger-zone-collections-6737118adcf2cb40d66d0c7e
->LoRA Spaces: https://huggingface.co/collections/prithivMLmods/lora-space-collections-6714b72e0d49e1c97fbd6a32
.
.
.
@prithivMLmods

darksfx hypergod ai4life44 rdrede John6666 Ngrthm MefhigosetH Tanvir1337 RenderIo


🏗 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲

🏗 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲

Watermark Anything with Localized Messages (2411.07231)

𝗠𝗲𝘁𝗮 𝘁𝗲𝗮𝗺 𝗷𝘂𝘀𝘁 𝗱𝗿𝗼𝗽𝗽𝗲𝗱 𝘁𝗵𝗲 𝗳𝗶𝗿𝘀𝘁 𝗪𝗮𝘁𝗲𝗿𝗺𝗮𝗿𝗸𝗶𝗻𝗴 𝗺𝗼𝗱𝗲𝗹 𝘁𝗵𝗮𝘁 𝗻𝗼𝘁 𝗲𝗱𝗶𝘁 𝗰𝗮𝗻 𝗯𝗿𝗲𝗮𝗸!🛡

🤔 Ever heard of watermarking? It's a technique that allows you to mark in an image its original source. It's our best shield against AI-generated deepfakes, or content stolen from artists! 🎨

🎭 Watermarking systems are actually a pair of models: a watermark embedder that applies the watermark on the image, and its corresponding decoder that should detect the original watermark.

⛔ But current methods were very limited: they can only apply and detect the watermark on your image as a whole. So, if you're an attacker it's easy to break: just crop it! add text on top! or whatever, really, anything would work to break the watermark.

A team of researchers at Meta was not happy with this. 😤

So to withstand real-world attacks, they decided to make a watermarking model that would also work on any sub-part of the image. It's a real paradigm shift: they consider watermarking not as an image classification task, but as an image segmentation task!

🏗 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲
▸ The "Embedder" (a variational autoencoder + embedder, 1.1M parameters in total) encodes a n-bit message into a watermark signal that is added to the original image
▸ [Only during training] The "Augmenter" randomly distorts the image: masks parts, crops, resizes, compresses. It's basically torture at this point.
▸ The "Extractor" (a vision transformer, or ViT, with 96M parameters) then re-extracts the message from the distorted image, by predicting a (1+n) vector per pixel to predict the watermarked parts and decode corresponding messages.

The performance blows existing models out of the water, they even created new tasks (segmentation-related) just to grok them!

Gerat work @pierrefdz and @tomsander1998!

Paper here 👉 https://huggingface.co/papers/2411.07231


Added @amphion MaskGCT & @hexgrad StyleTTS fine tuned model by the name of kokoro to the forked TTS Arena Space. If things keep up from what is seen in the preliminary results, then these two may end up in the TOP 5 of all TTS models. 🤞🍀

https://huggingface.co/spaces/Pendrokar/TTS-Spaces-Arena
https://huggingface.co/spaces/Svngoku/maskgct-audio-lab
https://huggingface.co/spaces/hexgrad/Kokoro-TTS

I chose @Svngoku 's forked HF space over amphion's due to the overly high ZeroGPU duration demand on the latter. 300s!

https://huggingface.co/spaces/amphion/maskgct

Had to remove @mrfakename 's MetaVoice-1B Space from the available models as that space has been down for quite some time.  🤕

https://huggingface.co/spaces/mrfakename/MetaVoice-1B-v0.1

I'm close to syncing the code to the original Arena's code structure. Then I'd like to use ASR in order to validate and create synthetic public datasets from the generated samples. And then make the Arena multilingual, which will surely attract quite the crowd!


Discovered an outrageous bug on the ChatGPT official website, especially for those using ad-blocking plugins. Please make sure to add `browser-intake-datadoghq.com` to your ad block whitelist. The ChatGPT webpage relies on this site for heartbeat detection, but since it belongs to an ad tracking network, it's included in major ad-blocking lists. (If you're using Clash, also remember to add it to the whitelist.) Failing to do so may cause the ChatGPT web interface to display a greyed-out send button after clicking, with no response.

For users with Chinese IP addresses, consider adding this URL to the rules of your U.S. node, as the response headers from this site will report the user's physical location to GPT.

aust-t dingangui csabakecskemeti Csplk ai-everyday


🎉 Celebrating One Year of #SauerkrautLM with Two Groundbreaking Releases!

We're thrilled to announce the release of SauerkrautLM-v2-14b in two specialized versions: https://huggingface.co/VAGOsolutions/SauerkrautLM-v2-14b-SFT and https://huggingface.co/VAGOsolutions/SauerkrautLM-v2-14b-DPO. Built on the robust Qwen2.5-14B foundation, these models represent a significant leap forward in multilingual AI capabilities.

🔬 Technical Breakthroughs:
💠 Innovative three-phase Fine-Tuning approach
💠 Two-step Spectrum SFT + one-step Spectrum DPO optimization phase for enhanced performance
💠 Balance of German and English language capabilities
💠 Advanced function calling - almost on par with Claude-3.5-Sonnet-20240620

🇩🇪 German Language Excellence:
What sets this release apart is our unique achievement in simultaneously improving both German and English capabilities. Through our specialized training approach with over 1.2B tokens across two phases, we've managed to:
💠 Enhance German language understanding and generation (SFT Version > DPO Version)
💠 Maintain authentic German linguistic nuances
💠 Improve cross-lingual capabilities
💠 Preserve cultural context awareness

📊 Training Innovation:
Our three-phase approach targeted specific layer percentages (15%, 20% and 25%) with carefully curated datasets, including:
💠 Mathematics-focused content (proprietary classifier-selected)
💠 High-quality German training data
💠 Specialized function calling datasets
💠 Premium multilingual content

🎁 Community Contribution:
We're also releasing two new datasets in a few days:
1⃣ SauerkrautLM-Fermented-GER-DPO: 3,300 high-quality German training samples
2⃣ SauerkrautLM-Fermented-Irrelevance-GER-DPO: 2,000 specialized samples for optimized function call  irrelevance handling

Thank you to our incredible community and partners who have supported us throughout this journey. Here's to another year of AI innovation! 🚀

rizky-gumelar ZeroXClem John6666 foscraft djuna not-lain


Early Morning Before Work Project:

🌌 Introducing Cascade of Semantically Integrated Layers (CaSIL): A Humorously Over-Engineered Algorithm That Actually… Works 🤷♂

Let me introduce CaSIL – the Cascade of Semantically Integrated Layers. Imagine giving a single question the level of introspection typically reserved for philosophical debates or maybe therapy. In short, CaSIL is a pure Python reasoning algorithm that, in a series of semantically rich layers, takes any input and rebuilds it into a nuanced response that’s (surprisingly) meaningful to a human.

I’ve been experimenting with various reasoning and agent approaches lately and decided to contribute my own quirky take on layered processing. It’s built without agent frameworks—just good ol' Python and math—and it plays nicely with any LLM. The result? A transformation from simple responses to deeper, interconnected insights. Here’s a quick peek at the steps:

✨ How CaSIL Works:

Initial Understanding: The first layer captures the basic concepts in your input, just as a warm-up.

Relationship Analysis: A lightweight knowledge graph (because why not?) maps out related ideas and builds interconnections.

Context Integration: Adds historical or contextual knowledge, bringing a bit of depth and relevance.

Response Synthesis: Pieces it all together, aiming to produce a response that feels more like a conversation than an outdated search result.

Does it work? Yes! And in record time, too. Admittedly, the code is rough—two days of intense coding with some friendly help from Claude. The beauty of CaSIL is its simplicity and versatility; it’s a pure algorithm without complex dependencies, making it easy to integrate into your own LLM setups.

🔗 Explore the repo here: https://github.com/severian42/Cascade-of-Semantically-Integrated-Layers

📜 Example outputs: https://github.com/severian42/Cascade-of-Semantically-Integrated-Layers/blob/main/examples.md


I just shipped `retrain-pipelines 0.1.1` today. The doc is also pimped compared to previous release. That was clearly not mature then.
I'll have to focus on another project for the next couple weeks but, anyone feel free to open issues on the GitHub repo and discuss any interest you'd have there if you will (please?) !
In the meantime, you may enjoy retrying this :
https://huggingface.co/blog/Aurelien-Morgan/stateful-metaflow-on-colab


🙋🏻♂hey there folks,

periodic reminder : if you are experiencing ⚠500 errors ⚠ or ⚠ abnormal `spaces`  behavior on load or launch ⚠

we have a thread 👉🏻 https://discord.com/channels/879548962464493619/1295847667515129877

if you can record the problem and share it there , or on the forums in your own post , please dont be shy because i'm not sure but i do think it helps 🤗🤗🤗

prithivMLmods Nymbo John6666 AtAndDev clem eyov Zaws


New Style, New Mix, New Drop 🧤

🧨Flux LoRA DLC: https://huggingface.co/spaces/prithivMLmods/FLUX-LoRA-DLC

🎆Glowing-Body: https://huggingface.co/prithivMLmods/Glowing-Body-Flux-LoRA
🎆Electric-Blue: https://huggingface.co/prithivMLmods/Electric-Blue-Flux-LoRA
🎆Intense-Red: https://huggingface.co/prithivMLmods/Intense-Red-Flux-LoRA
🎆Clouds-Illusion: https://huggingface.co/prithivMLmods/Clouds-Illusion-Flux-LoRA
🎆Digital-Yellow: https://huggingface.co/prithivMLmods/Digital-Yellow-Flux-LoRA

🧨Flux LoRA Collection: https://huggingface.co/collections/prithivMLmods/flux-lora-collections-66dd5908be2206cfaa8519be

.
.
.
@prithivMLmods

Tonic realaliarain speedchemistry JayNagose prithivMLmods multimodalart radames d8rt8v Hamed744 rdrede hypergod darksfx ai4life44 faruqhrp Ngrthm RenderIo

jovialjoel kimp-dev-ninja rdrede zaidzameer010 ai4life44 darksfx hypergod prithivMLmods Ngrthm RenderIo

John6666 speedchemistry Markjr multimodalart rdrede Rsln prithivMLmods Ngrthm

Tonic multimodalart rdrede hypergod ai4life44 prithivMLmods Ngrthm

rdrede hypergod darksfx prithivMLmods louisbrulenaudet Ngrthm


Vector Search (most) datasets on the Hugging Face Hub 🔦

Powered by: Polars, DuckDB, Gradio and model2vec (lightning-fast embeddings by Stéphan Tulkens).

Should work fast enough for datasets up to 100K.

https://huggingface.co/spaces/davidberenstein1957/vectorsearch-hub-datasets

davidberenstein1957 Tonic prithivMLmods xi0v Nymbo djuna thanhkt


Hello researchers! Here are scripts to generate reviews on HF Daily Papers:

👉 https://github.com/averkij/top_papers

⚙ Works on GitHub Actions
🤖 Claude, GPT-4o, FLUX
🌏 Multiple languages
📚 Classification by 38 topics (#agents, #multimodal, #plp, etc.)
🔺 https://HFday.ru


It's work like this that in some way signal the eventual “dominance” of AI over all the sciences.

“We train our model on the six-dimensional N-body phase space, predicting particle velocities as the time derivative of the model’s displacement outputs”

The emulator is capable of predicting
the nonlinear displacement and velocity fields for 128^3 particles in half a second on a single GPU🤯


I've released several new Hugging Face Spaces.

My primary objective is to create consistent character facial animation using image-to-image techniques:

https://huggingface.co/spaces/Akjava/CreateConsistentCharacterFacialAnimationWithImg2Img

A short-term goal is create simple talk-head animation.

WebP-3-Frame-Talking-Animation
https://huggingface.co/spaces/Akjava/AIDiagramChatWithVoice-FaceCharacter

[Space]

- GPU tools
Flux1-schnell img2img
https://huggingface.co/spaces/Akjava/flux1-schnell-img2img

Flux1-schnell Inpaint with mask-file
https://huggingface.co/spaces/Akjava/flux1-schnell-img2img

 - Tiny CPU tools
WebP-3F-TH - create webp animation from 3 images
OpenCV-Inapint - classic inpaint
Whitebalance - simple white balance
Paste Image - just paste image with mask
WebP Resize Convert - resize and convert webp-animation


Forget any document retrievers, use ColPali 💥💥

Document retrieval is done through OCR + layout detection, but you are losing a lot of information in between, stop doing that! 🤓

ColPali uses a vision language model, which is better in doc understanding 📑
ColPali: https://huggingface.co/vidore/colpali (mit license!)
Blog post: https://huggingface.co/blog/manu/colpali
The authors also released a new benchmark for document retrieval:
ViDoRe Benchmark: https://huggingface.co/collections/vidore/vidore-benchmark-667173f98e70a1c0fa4db00d
ViDoRe Leaderboard: https://huggingface.co/spaces/vidore/vidore-leaderboard

ColPali marries the idea of modern vision language models with retrieval 🤝

The authors apply contrastive fine-tuning to SigLIP on documents, and pool the outputs (they call it BiSigLip). Then they feed the patch embedding outputs to PaliGemma and create BiPali 🖇
BiPali natively supports image patch embeddings to an LLM, which enables leveraging the ColBERT-like late interaction computations between text tokens and image patches (hence the name ColPali!) 🤩

The authors created the ViDoRe benchmark by collecting PDF documents and generate queries from Claude-3 Sonnet.
ColPali seems to be the most performant model on ViDoRe. Not only this, but is way faster than traditional PDF parsers too!

ucsahin louisbrulenaudet Jebadiah diasbalmash osanseviero Zhofang


LivePortrait: Efficient Portrait Animation with Stitching and
  Retargeting Control (2407.03168)

🔥🎭🌟 New Research Alert - LivePortrait (Avatars Collection)! 🌟🎭🔥
📄 Title: LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control 🔝

📝 Description: LivePortrait is an efficient video-driven portrait animation framework that uses implicit keypoints and stitching/retargeting modules to generate high-quality, controllable animations from a single source image.

👥 Authors: @cleardusk, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang

🤗 Demo: https://huggingface.co/spaces/KwaiVGI/LivePortrait

📄 Paper: https://huggingface.co/papers/2407.03168

🌐 Github Page: https://liveportrait.github.io/
📁 Repository: https://github.com/KwaiVGI/LivePortrait

🔥 Model 🤖: https://huggingface.co/KwaiVGI/LivePortrait

🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers

🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers

🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #LivePortrait #PortraitAnimation #ComputerVision #MachineLearning #DeepLearning #ComputerGraphics #FacialAnimation #GenerativeAI #RealTimeRendering #AI

DmitryRyumin Andzej-75 bobros dessignAI cleardusk jimdigitart GPT007 Blucky aoezdTchibo victor

DmitryRyumin Shinku Andzej-75 cleardusk osanseviero


Hello.

I'm excited to announce that 'LivePortrait' has been released, and we've developed an advanced application service using it. We've made this service available for free on our Discord server (https://discord.gg/openfreeai). To achieve this, we've deployed four Nvidia H100 GPUs and optimized our code for multi-GPU processing.

On our Discord server, we've named this channel "Mimic Face." This incredible free service allows you to create an interactive video in just a few seconds based on your photo. Simply upload a photo and select a face video you want to mimic. The AI will then replicate your face, mimicking gestures, expressions, mouth movements, eye movements, and even blinking naturally.

The process is extremely simple, and the results are impressively realistic. Experience this fantastic service for yourself, and stay tuned for more exciting AI services we plan to introduce in the future.

Open Service link: https://discord.gg/openfreeai

Channel Direct Link: https://discord.com/channels/1228254992729767996/1259659402118692896

aiqtech fantaxy fantos ginipick MaziyarPanahi julien-c victor Ramikan-BR jimdigitart seawolf2357

aiqtech fantaxy fantos ginipick Alpaggos julien-c Ramikan-BR

aiqtech fantaxy Blane187 julien-c victor Ramikan-BR seawolf2357


# LivePortrait 1-Click Installers and full tutorials for Windows and Cloud (useful for Mac users) (Massed Compute, RunPod and a free Kaggle Account)

I know there are a lot of experts around here that can install and use easily. But I have prepared solid tutorials for newbies and shown how to use this amazing top quality app LivePortrait. I have to say congrats to the developers.

I am also a researcher you can see my LinkedIn profile here but recently I am shifted into AI lectures : https://www.linkedin.com/in/furkangozukara/

Both Windows and Cloud tutorial has manually written (100% accurate) captions / subtitles. Also both have manually written by me very detailed video chapters.

Windows LivePortrait Tutorial : https://youtu.be/FPtpNrmuwXk

Cloud LivePortrait Tutorial : Massed Compute, RunPod & Kaggle : https://youtu.be/wG7oPp01COg

## Windows LivePortrait Tutorial Video Chapters

- 0:00 Introduction to LivePortrait: A cutting-edge open-source application for image-to-animation conversion
- 2:20 Step-by-step guide for downloading and installing the LivePortrait Gradio application on your device
- 3:27 System requirements and installation process for LivePortrait
- 4:07 Verifying the successful installation of required components
- 5:02 Confirming installation completion and preserving installation logs
- 5:37 Initiating the LivePortrait application post-installation
....


🚨 New Release: ultralytics8.2.51
🍺Live Space : https://huggingface.co/spaces/prithivMLmods/YOLO-VIDEO , Duplicate the Space to avoid queuing issues.
🍺T4  Colab : https://colab.research.google.com/drive/1BKgFUfk2Me1cSPFmbtZSVCn_4cYImPO-?au
👉🏻For HPC, use A100/T4 under controlled conditions.
👉🏻Speed Estimation, Object Counting, Distance Calculation, Workout Monitoring, Heatmaps,etc.

Ultralytics dropped the YOLOv8 - #Ultralytics 8.2.51 🔥, YOLOv8 is designed to be fast, accurate, and easy to use, making it an excellent choice for a wide range of object detection and tracking, instance segmentation, image classification and pose estimation tasks.

🔗 https://pypi.org/project/ultralytics/8.2.58/

🚀More Features You can try:
✅ Classes selection support added
✅ Live FPS display in the sidebar
✅ Webcam and video support added
✅ Confidence and NMS threshold option to modify.
✅ Segmentation, detection, and pose models support added.

🙀Ultralytics Live inference: https://docs.ultralytics.com/guides/streamlit-live-inference/
```
from ultralytics import solutions
solutions.inference()
### Make sure to run the file using command `streamlit run <file-name.py>`

```
⚡yolo streamlit-predict

👉🏻Advantages of Live Inference

      ☑ Seamless Real-Time Object Detection: Streamlit combined with YOLOv8 enables real-time object detection directly from your webcam feed. This allows for immediate analysis and insights, making it ideal for applications requiring instant feedback.
      ☑Efficient Resource Utilization: YOLOv8 optimized algorithm ensure high-speed processing with minimal computational resources.
🙀Ultralytics feature Models: https://docs.ultralytics.com/models/, Ultralytics new Solutions: https://docs.ultralytics.com/solutions/

👉🏻Official Documentation:
    Ultralytics YOLOv8 Documentation: Refer to the official YOLOv8 documentation for comprehensive guides and insights on various computer vision tasks and projects. 🔗 https://docs.ultralytics.com/


Check @lllyasviel Latest research for making videos of still images using hand drawing keyframes:
https://lllyasviel.github.io/pages/paints_undo/

I imported his work to gradio space here
https://huggingface.co/spaces/MohamedRashad/PaintsUndo


ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wild (2407.04172)

📢 Exciting News! Our latest paper "ChartGemma" is out! 📊

🧵1/3: ChartGemma overcomes existing chart models key limitations that rely too much on data tables. Instead, it is trained on data generated directly from chart images, capturing crucial visual trends📸🔍

🧵2/3: ChartGemma builds upon PaliGemma from Google Research and is fine-tuned on a high-quality visual instruction tuning dataset generated from Gemini Flash 1.5. 🌟📊

🧵3/3: Achieves state-of-the-art results in chart summarization, question answering, and fact-checking tasks. 🏅📊 It can also generate more accurate and realistic chart summaries. 📝🔍

Our model and data are publicly available. We also have a cool web demo. Check it out! 🚀
Demo: https://huggingface.co/spaces/ahmed-masry/ChartGemma
Code: https://github.com/vis-nlp/ChartGemma
Paper: https://huggingface.co/papers/2407.04172

megh1211 osanseviero nbroad ucsahin GPT007 John6666 damerajee AtAndDev FedericoVal holooo shivanshdhar


How's 2024 been so far? 🎉 Well, we've got the data to find out!

New dataset just dropped on the Hub: 12.5 million Reddit comments from 2024! 📊🗣

Features:
- Personal info anonymized 🕵♀
- Language detection 🌐
- Token count 🔢
- NSFW filtering 🚫

Most popular post? Classic AITA drama: "AITA for 'ruining Christmas' and being upset the only gifts I got from my family were 'joke gifts'" 🎄😅

Some things never change, huh?

Dive in: https://huggingface.co/datasets/OpenCo7/UpVoteWeb


Just updated the Journalists on 🤗 community with two new AI tools! 🚀📊

Check them out:
- Free video transcription & smart summary tool https://huggingface.co/spaces/artificialguybr/Video-Transcription-Smart-Summary
- ChartGemma - next-level chart analysis focusing on visual trends https://huggingface.co/spaces/ahmed-masry/ChartGemma

Media folks: Join our community for more tools! https://huggingface.co/JournalistsonHF


Chrome's new `window.ai` feature is going to change the web forever! 🤯 It allows you to run Gemini Nano, a powerful 3.25B parameter LLM, 100% locally in your browser!

We've also added experimental support to 🤗 Transformers.js!
- Demo: https://huggingface.co/spaces/Xenova/experimental-built-in-ai-chat
- Blog post: https://huggingface.co/blog/Xenova/run-gemini-nano-in-your-browser

victor clem GPT007 John6666 shreyask prithivMLmods indigoochoa osanseviero nbroad ucsahin MoritzLaurer eryk-mazus KwabsHug KaifResearch

clem GPT007 shreyask osanseviero louisbrulenaudet jrwren


Hi HF community! 🤗

There's a new space out in the wild! https://huggingface.co/spaces/as-cle-bert/self-reviewing-coding-assistant 🦜

It's a self-correcting and self-reviewing python coding assistant based on GPT4-o and LangChain, inspired by Codium-AI's AlphaCodium 👾

Have fun! 🐙


One more cookbook:
Agent for self-correcting Text-to-SQL 🧑💻

What if the query generated by your Text-to-SQL pipeline is correct SQL but returns wrong results?
👉 We need to add a critique step

✅ That's very simple with an agent!

Check out the notebook! 👇
https://huggingface.co/learn/cookbook/agent_text_to_sql


Is a full-featured Mac sufficient for AI and ML development compared to NVIDIA GPU systems?

Hello everyone,
We are about to make the decision to purchase a powerful Mac with maximum features for our university. This Mac will primarily serve as a development computer in the field of artificial intelligence (AI) and machine learning (ML) and will be used by a small group of users in the local network. After development, the systems will be transferred to servers that have to withstand higher loads and visitor numbers.

Planned system:
Apple Mac Studio 2023 with M2 Ultra processor
24-core CPU
76-core GPU
32-core NPU (neural engine) for machine learning
128GB RAM
1TB HDD

Our question to you:
Is a fully equipped Mac with the latest SoCs chips and integrated neural engines sufficient for the development of AI and ML systems?
Or should we rather rely on proven Windows/Linux systems with powerful NVIDIA graphics cards?

We already have several NVIDIA graphics cards available at the university:
NVIDIA Tesla T4
NVIDIA 2080Ti
NVIDIA 3080Ti

We are particularly interested in your experiences and assessments of how the performance of the Mac compares to the GPU systems mentioned.

Are there significant differences, especially in the development and training of models?

What difference would you consider to be significant?

For us, a difference of 100% or more would be considered significant.
In other words, computer A (Mac) takes twice as long to calculate as computer B (NVIDIA system).

Many thanks in advance for your answers and experience!
Best regards,

Oliver


Hi everyone,

I am excited to introduce our latest work, LLaMAX.  😁😁😁

LLaMAX is a powerful language model created specifically for multilingual scenarios. Built upon Meta's LLaMA series models, LLaMAX undergoes extensive training across more than 100 languages.

Remarkably, it enhances its multilingual capabilities without compromising its generalization ability, surpassing existing LLMs.

✨Highlights:

🎈 LLaMAX supports the 102 languages covered by Flores-101, and its performance in translating between low-resource languages far surpasses other decoder-only LLMs.

🎈 Even for languages not covered in Flores-200, LLaMAX still shows significant improvements in translation performance.

🎈 By performing simple SFT on English task data, LLaMAX demonstrates impressive multilingual transfer abilities in downstream tasks.

🎈 In our paper, we discuss effective methods for enhancing the multilingual capabilities of LLMs during the continued training phase.

We welcome you to use our model and provide feedback.

More Details:

🎉  Code:  https://github.com/CONE-MT/LLaMAX/

🎉  Model:   https://huggingface.co/LLaMAX/

yqlu LLaMAX John6666 boapps Ramikan-BR GPT007 qnguyen3 Joseph717171 htafer osanseviero paulml ruben-wleon kramp apol

Gutssu Ramikan-BR GPT007 prithivMLmods yoeldcd Joseph717171 osanseviero juancopi81 MikeMpapa

Symbol-LLM Ramikan-BR GPT007 Joseph717171 ucsahin FilipeR


Hey everyone,

We're thrilled to introduce our latest project: a hand-curated list of the best-curated lists related to artificial intelligence! 🚀

Check it out here: https://github.com/zhimin-z/awesome-awesome-artificial-intelligence


📍Excited to make public a series of checkpoints !

- Final checkpoints after self-training with ENVISIONS framework
- Cover math, logic, and agent domains
- Include 7B / 13B

📕 Check our paper:
Title: Interactive Evolution: A Neural-Symbolic Self-Training Framework For Large Language Models
Link: https://arxiv.org/abs/2406.11736


Please, help with is!!!! "You have exceeded your GPU quota... " I have paid the Pro service of 9 Euros per month, and everything remains the same, I barely do four tasks and I get the restriction message that I have to wait many minutes or more than an hour. So why have I paid the 9 Euros a month, if everything remains the same, with the same restrictions as if it were a free account? I have written, but they do not respond to me. Please help me.


Newswire: A Large-Scale Structured Database of a Century of Historical
  News (2406.09490)

Very cool dataset for journalists and historians just dropped: 2.7 million unique public domain U.S. news wire articles (1878-1977) 📰🕰

This is a goldmine for tracking historical events & newspaper coverage trends! An example? If we still wonder whether gender diversity in the media is important... "Only 4.6% of disambiguated entity mentions refer to women, and the most mentioned woman is Golda Meir."

Bonus:
- Locations in these articles are georeferenced
- Topics are tagged using customized neural topic classification
- Named entities are recognized,
- Individuals are disambiguated to Wikipedia using a novel entity disambiguation model

Anyone thinking of cool AI projects with this data? Maybe tracking the spread of news stories over time & space?

𝐆𝐨 𝐝𝐞𝐞𝐩𝐞𝐫
👉 Digg into the dataset: https://huggingface.co/datasets/dell-research-harvard/newswire
 👉 Read the paper: https://huggingface.co/papers/2406.09490


Running billion parameter models, sometimes we forget what it all is! 🤔💡

Matrix multiplication 🧮✨

While there are multiple plays on memory management and caching to speed it up! 🏎💾⚡

The naive way of Matrix multiplication becomes even more fascinating the bigger these models get! 🤯📈

QKV for the win! 🏆🔑📚

GitHub: https://github.com/wentasah/mmul-anim
Slides: https://cw.fel.cvut.cz/wiki/_media/courses/b4m36esw/esw09_2019.pdf 📑🎓


Introducing the first two projects on the HFforLegal community: the 'Laws' dataset and the associated search tool based on @nreimers and @tomaarsen's Sentence Transformers library 🤗

The objective of these two tools is to centralize in a single format a set of rules from different countries and legal systems in order to facilitate NLP in the field of comparative law, enabling more accurate and comprehensive legal analysis across different jurisdictions 🌍

Link to the dataset : https://huggingface.co/datasets/HFforLegal/laws
Link to the space: https://huggingface.co/spaces/HFforLegal/laws-retrieval

We need your contributions to enrich this new knowledge base, and you will find in the 'Laws' dataset all the information you need to format your data and submit them to the appropriate split.


New cookbook!

I show to to make agentic RAG using Transformers Agents.

Compared to vanilla RAG, agentic RAG can:
✅ Reformulate the query
✅ Critique the retrived content to re-retrieve if needed

➡ Score increase of 8.5%! 💪 (Llama-3-70B-judge)

Read it here 👉 https://huggingface.co/learn/cookbook/agent_rag


Animate a portrait with a driving video. Lots of potential fun here 😅 https://huggingface.co/spaces/KwaiVGI/LivePortrait

prithivMLmods Ramikan-BR GPT007 victor osanseviero Yersel


Kolors with VLM support

I've built a space for using Kolors image generation model with captioner models and prompt enhancers.

- Space with VLM and Prompt Enhancer
 https://huggingface.co/spaces/gokaygokay/KolorsPlusPlus

- Original Space for model
https://huggingface.co/spaces/gokaygokay/Kolors

- Captioner VLMs
- https://huggingface.co/gokaygokay/sd3-long-captioner-v2

- https://huggingface.co/microsoft/Florence-2-base

- Prompt Enhancers
- https://huggingface.co/gokaygokay/Lamini-Prompt-Enchance-Long

- https://huggingface.co/gokaygokay/Lamini-Prompt-Enchance

John6666 ucsahin kramp osanseviero Ramikan-BR yoeldcd tenet Joseph717171 AtAndDev Wok


Hey everyone!

Our team just dropped something cool! 🎉 We've published a new paper on arxiv diving into the foundation model leaderboards across different platforms. We've analyzed the content, operational workflows, and common issues of these leaderboards. From this, we came up with two new concepts: Leaderboard Operations (LBOps) and leaderboard smells.

We also put together an awesome list with nearly 300 of the latest leaderboards, development tools, and publishing organizations. You can check it out here: https://github.com/SAILResearch/awesome-foundation-model-leaderboards

If you find it useful or interesting, give us a follow or drop a comment. We'd love to hear your thoughts and get your support! ✨

Link to the paper: https://arxiv.org/abs/2407.04065

zhiminy osanseviero louisbrulenaudet Tonic


Excited to announce the release of the community version of our guardrails: WalledGuard-C!

Feel free to use it—compared to Meta’s guardrails, it offers superior performance, being 4x faster. Most importantly, it's free for nearly any use!

Link: https://huggingface.co/walledai/walledguard-c

#AISafety

Mehyaar osanseviero RishabhBhardwaj prithivMLmods


Anyone knows how to use SD3 for inpaint task ? :)


BrainGPT - Fun Weekend Project:)
Getting creative with a sci-fi 3D point cloud model of the brain - you prompt the model with questions about AI research frameworks that were deeply inspired by parts of the brain, you get a response with related papers 😂


Yet Another Whisper Finetune

https://huggingface.co/MohamedRashad/Arabic-Whisper-CodeSwitching-Edition is a finetune of OpenAI Whisper Large V2 on https://huggingface.co/datasets/MohamedRashad/arabic-english-code-switching dataset.
This new finetune is capable of recognizing english words in arabic speech and transcribing the foreign words as it is.

Try it out here:
https://huggingface.co/spaces/MohamedRashad/Arabic-Whisper-CodeSwitching-Edition

MohamedRashad osanseviero nyuuzyou manoumhd99


Introducing: The P-FAF Swarm Encoder! Now that I know you can do it with Swarm algorithms, the sky is the limit, beebee! The hotness AI tools over the past month have been LLM models with self referential feedback. AKA, getting them to do stuff on their own, like create data, transform data, etc. What if I told you that was wholly inefficient to do? What if I could replicate the same things and run it on a calculator? Meet P-FAF Swarm Encoder, your first real world proof that this is beyond viable. Plenty more proof to come!

Colab Notebook to try it yourself:
https://colab.research.google.com/drive/14CRumDep0SEubEiG3DCOCuGNGeZcsUkO?usp=sharing


🤗 Hi HF community!
🤖 A vital question that every developer may have asked themselves in the last three years is: how can we improve AI code generation?
😇 In my last Community blog post, I talk about  Codium AI's AlphaCodium and how they tried to enhance LLMs coding  skills with flow engineering: https://huggingface.co/blog/as-cle-bert/repetita-iuvant-how-to-improve-ai-code-generation
🚀 Enjoy!


A new paper titled "STALL+: Boosting LLM-based Repository-level Code Completion with Static Analysis" shows the benefits of integrating static analysis with LLMs. (https://arxiv.org/abs/2406.10018)

Authors evaluate 4 key questions:

- How does each static analysis integration strategy perform in LLM-based repository-level code completion?
> They found that integrating static analysis in the prompting phase (especially with file-level dependencies) can achieve the substantially larger improvements than other phases.

- How do different combinations of integration strategies affect LLM-based repository-level code completion?
> Languages that are easier to analyze like Java show more improvements compared to dynamic languages like Python.

- How do static analysis integration strategies perform when compared or combined with RAG in LLM-based repository-level code completion?
> Static analysis and RAG are complementary and boost the overall accuracy.

-  What are the online costs of different integration strategies in LLM-based repository-level code completion?
> Combining prompting-phase static analysis and RAG is the best option for cost-effectiveness.

In my @owasp App Sec keynote last year, I had described how one can do static analysis augmented generation (SaAG) to boost the accuracy of LLM based patches for vulnerability remediation. (you can see the talk here - https://www.youtube.com/watch?v=Cw4-ZnUNVLs)


Caffe 2 started it, TensorFlow brought it to the masses but PyTorch perfected it! ✨

Good folks at PyTorch are just wizards 🧙♂. Along with producing one of the best Deep Learning libraries of all time, they just dropped the "PyTorch Documentary" 🎥

A must-watch! It covers the beginning to now:
- Caffe 2 ☕
- Torch (with Lua) 🔥
- Tensorflow 🔄
- PyTorch 🔥
- Taffe IR (later became ONNX - Open Neural Network Exchange) 🔧

Full Official PyTorch Documentary: Powering the AI Revolution: https://youtu.be/rgP_LBtaUEc

Interesting quote: "PyTorch does not fight for the fastest performance but the easiest user experience!" 🌟

That's what Python 🐍 feels like...

You got to thank @Meta for open-sourcing it! 🤝


🧠 How to create more diverse, realistic synthetic AI training data?

@TencentAIGC-Lab AI Lab created @proj-persona, a vast collection of 1 billion diverse personas, to help create synthetic data with LLMs that encapsulate a wide array of perspectives, knowledge, experiences, interests, and professions.

These personas were created with automatically curated data, representing approximately 13% of the world’s total population.

💡 The authors argue that integrating a persona into data synthesis prompts effectively steers LLMs to adopt specific perspectives, creating unique and relevant synthetic data with minimal effort.

They showcased various practical applications of Persona Hub to demonstrate its effectiveness and versatility in various synthetic data creation scenarios: mathematical and logical reasoning problems, simulating diverse user requests and prompts for LLMs, generating informative and detailed text content across various topics, and more.

🚀 It's one of the trending datasets on Hugging Face. Digging into it is quite fun! I found one that reminds me of several people I know: "A journalist who covers technology and innovation in the print and digital media industries." It helped generate the prompt attached to this post (about which I'd be curious to know your answers 😉).

Synthetic data is a hot topic in AI. It will be interesting to see if this research could help make LLMs more robust, versatile, and capable of handling a wide array of real-world scenarios.

👉Explore the dataset: https://huggingface.co/datasets/proj-persona/PersonaHub
👉 Read the paper: https://arxiv.org/pdf/2406.20094

proj-persona ZeroWw mahami John6666 vikas louisbrulenaudet samsen5


I updated https://huggingface.co/spaces/nroggendorff/llava to be even worse. The point of this space is to point out how bad the base model for Llava is, and how without an image it struggles quite a bit. In the new update, from my testing, even if you ask the model about the image, it won't be able to tell you. I experimented with quite a few things, including an image where all the values are 0 (shoutout np.zeros), an image with the most generic portrait photo I could think of (black hair, brown eyes, plain white shirt, etc..) (generated with https://huggingface.co/spaces/nroggendorff/epicrealismxl), an image that has text that reads "The image you are looking for is unavailable", and much.. much more.. *sigh*. The image I'm ending up with seems to work best. I don't think it would work with other multi-modal LLMs, but I'm not sure. If you find a problem, feel free to message me on the [huggingface discord](http://hf.co/join/discord), or open a pull request.


📢 Delighted to share the most recent and valuable contributions to the book-related NLP domain 💎 To push forward deeper understanding of the characters 👨👩👧👦 from literature novel books itself 📖 by machine learning models 🤖, releasing the most-accessible version v1.0 of the related workflow, adopted for ParlAI 🦜 agents

🌟 https://github.com/nicolay-r/book-persona-retriever/tree/v1.0

Feel free to follow / share / comment in order to advance the related direction!


AI’s Cognitive Mirror: The Illusion of Consciousness in the Digital Age

https://empereur-pirate.medium.com/ais-cognitive-mirror-the-illusion-of-consciousness-in-the-digital-age-46f3ddae60a6

This text explores the philosophical implications of AI's potential for spiritual consciousness. It argues that AI, while capable of complex thought processes, lacks true self-awareness due to the absence of sensory perception. The article discusses how human consciousness develops through sensory experiences, contrasting this with AI's purely computational nature. It examines AI as a mirror of human thought, capable of impressive language processing and knowledge synthesis, yet  different from human psyche. The text also touches on the spiritual and ethical dimensions of AI development, drawing parallels with historical technological revolutions and religious symbolism. It concludes by considering AI's potential role in decision-making processes and its impact on various fields, while emphasizing the need for ethical safeguards and critical evaluation of AI-generated content.


Just released the GitVerse Code Dataset - https://huggingface.co/datasets/nyuuzyou/gitverse-code.

📊 Dataset highlights:
- 30 GB of unique code extracted from over 400 GB of analyzed data
- 9,014 repositories
- 2,804,216 unique code files
- 419 different file types
- Multilingual: various programming languages

🌐 Sourced from GitVerse, a Russian GitHub alternative opened in 2024.

Let me know your thoughts.

ZeroWw JoPmt nhannguyen1992 DeathGodlike nyuuzyou


Kolors: Effective Training of Diffusion Model for Photorealistic Text-to-Image Synthesis

Kolors is a large-scale text-to-image generation model based on latent diffusion, developed by the Kuaishou Kolors team.

Hugging Face Spaces
- https://huggingface.co/spaces/gokaygokay/Kolors

Model Page
- https://huggingface.co/Kwai-Kolors/Kolors


Really got amazing results with our InstantID next level. Tutorial will be published soon hopefully. This is the best ever 0-shot likeliness and high quality I have ever got.

prithivMLmods GPT007 rreed-pha NHLOCAL cnmoro Kkordik


Remember the recently released GLM-4 from Tsinghua University 🎉

Now we have an open-source version of it continuously trained for multilingual code generation! 🌍

It beats CodeLlama 70B (almost 7x size) and is competitive with DeepSeek Coder 33B and Qwen 2 💪

Just like almost every other coding model, it has a 128K context 📜

Supports:
- Code completion 🖋
- Code generation 🛠
- Code interpreter 💡
- Web search 🔍
- Function call 📞
- Repository-level code Q&A 🗂

Benchmarks 48.9 and 40.4 for the complete and instruct tasks of BigCodeBench 📊

It still falls behind DeepSeek-Coder-V2. While this might have fewer parameters, DSC V2 is a MoE model with only ~2B active parameters 🤔

Good to see more efficient coding LLMs, but DeepSeek-Coder-V2 is just too good 🏆

Codel: https://github.com/THUDM/CodeGeeX4
Model weights: https://huggingface.co/THUDM/codegeex4-all-9b


I believe in order to make models reach Human-Level Learning, serious students can start by developing an intelligent neuromorphic agent. We develop an intelligent agent and make it learn about grammar patterns as well as about different word categories through symbolic representations, following which we dwell into making the agent learn about other rules of the Language.

In parallel with grammar learning, the agent would also use language grounding techniques to link words to their sensory representations and abstract concepts which would mean the agent learns about the word meanings, synonyms, antonyms, and semantic relationships from both textual data as well as perceptual experiences.

The result would be the agent developing a rich lexicon and conceptual knowledge base that underlies its language understanding as well as generation. With this basic knowledge of grammar and word meanings, the agent can then learn to synthesize words and phrases so as to express specific ideas or concepts. Building on this, the agent would then learn how to generate complete sentences which the agent would continuously refine and improve. Eventually the agent would learn how to generate sequence of sentences in the form of dialogues or narratives, taking into account context, goals, as well as user-feedback.

I believe that by gradually learning how to improve their responses, the agent would gradually also acquire the ability to generate coherent, meaningful, and contextually appropriate language. This would allow them to reason without hallucinating which LLMs struggle at.

Developing such agents would not require a lot of compute and the code would be simple & easy to understand. It will definitely introduce everyone to symbolic AI and making agents which are good at reasoning tasks. Thus solving a crucial problem with LLMs. We have used a similar architecture to make our model learn constantly. Do sign up as we start opening access  next week at https://octave-x.com/


ColPali: A new approach to efficient and intelligent document retrieval 🚀

Our latest research paper, "ColPali: Efficient Document Retrieval with Vision Language Models," introduces a groundbreaking approach to large-scale visual document analysis. By leveraging Vision Language Models (VLMs), we have created a new framework for document retrieval that's both powerful and efficient.

Key Insights:
💡 ColPali combines ColBERT's multi-vector strategy with VLMs' document understanding capabilities
⚙ ColPali is based on PaliGemma-3B (SigLIP, Gemma-2B) + a linear projection layer and is trained to maximize the similarity between the document and the query embeddings
📊 The Vision Document Retrieval benchmark (ViDoRe) is a challenging dataset that spans various industry topics and aims at matching real-life retrieval scenarios
🏆 ColPali outperforms existing models on all datasets in ViDoRe (average NDCG@5 of 81.3% vs 67.0% for the best baseline model)
⚡ ColPali is faster at document embedding compared to traditional PDF parser pipelines, making ColPali viable for industrial use
🔍 ColPali is highly interpretable thanks to patch-based similarity maps

Dive deeper into ColPali and explore our resources:
📑 Full paper: arxiv.org/abs/2407.01449
🛠 Datasets, model weights, evaluation code, leaderboard, demos: huggingface.co/vidore

Shoutout to my amazing co-authors Manuel Faysse (@manu) and Hugues Sibille (@HugSib). We are grateful for the invaluable feedback from Bilel Omrani, Gautier Viaud, Celine Hudelot, and Pierre Colombo. This work is sponsored by ILLUIN Technology. ✨


Expressive Gaussian Human Avatars from Monocular RGB Video (2407.03204)

🚀🕺🌟 New Research Alert (Avatars Collection)! 🌟💃🚀
📄 Title: Expressive Gaussian Human Avatars from Monocular RGB Video 🔝

📝 Description: The new EVA model enhances the expressiveness of digital avatars by using 3D Gaussians and SMPL-X to capture fine-grained hand and face details from monocular RGB video.

👥 Authors: Hezhen Hu, Zhiwen Fan, Tianhao Wu, Yihan Xi, Seoyoung Lee, Georgios Pavlakos, and Zhangyang Wang

📄 Paper: https://huggingface.co/papers/2407.03204

🌐 Github Page: https://evahuman.github.io/
📁 Repository: https://github.com/evahuman/EVA

🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers

🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers

🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #DigitalAvatars #3DModeling #ComputerVision #MonocularVideo #SMPLX #3DGaussians #AvatarExpressiveness #HandTracking #FacialExpressions #AI #MachineLearning

DmitryRyumin m1b prithivMLmods not-lain andmev Joe2EZ


MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via
  Dynamic Sparse Attention (2407.02490)

Weclome to use MInference, which leverages the dynamic sparse nature of LLMs' attention, which exhibits some static patterns, to speed up the pre-filling for million tokens LLMs. It first determines offline which sparse pattern each head belongs to, then approximates the sparse index online and dynamically computes attention with the optimal custom kernels. This approach achieves up to a 10x speedup for pre-filling on an A100 while maintaining accuracy with 1M tokens.

For more detail please check,
project page: https://aka.ms/MInference
code: https://github.com/microsoft/MInference
paper: https://huggingface.co/papers/2407.02490
hf demo: https://huggingface.co/spaces/microsoft/MInference


New dataset filtering feature just dropped! 🤗🚀

Find exactly what you need with filters for:
- Modalities (text, image, audio, etc.)
- Dataset size
- File format

Try it now: https://huggingface.co/datasets

What other filters would you find useful? Drop your ideas!


Hi @jonoirwin! Big fan of https://fastvoiceagent.cerebrium.ai/ 🔥
I'd be super happy to give you a GPU grant to host it on a Space, it would allow more people to discover and use it!

lunarflu RakshitAralimatti nyuuzyou Inflammable1230


I've created a space for chatting with Gemma 2 using llama.cpp

- 🎛 Choose between 27B IT and 9b IT models
- 🚀 Fast inference using llama.cpp

- https://huggingface.co/spaces/gokaygokay/Gemma-2-llamacpp

John6666 merterbak victor lunarflu thethinkmachine osanseviero ggerganov juancopi81 ucsahin nedegilefendim Metin


@Omartificial-Intelligence-Space has trained and released 6 Arabic embedding models for semantic similarity. 4 of them outperform all previous models on the STS17 Arabic-Arabic task!

📚 Trained on a large dataset of 558k Arabic triplets translated from the AllNLI triplet dataset: https://huggingface.co/datasets/Omartificial-Intelligence-Space/Arabic-NLi-Triplet
6⃣ 6 different base models: AraBERT, MarBERT, LaBSE, MiniLM, paraphrase-multilingual-mpnet-base, mpnet-base, ranging from 109M to 471M parameters.
🪆 Trained with a Matryoshka loss, allowing you to truncate embeddings with minimal performance loss: smaller embeddings are faster to compare.
📈 Outperforms all commonly used multilingual models like https://huggingface.co/intfloat/multilingual-e5-large, https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2, and https://huggingface.co/sentence-transformers/LaBSE.

Check them out here:
- https://huggingface.co/Omartificial-Intelligence-Space/Arabic-mpnet-base-all-nli-triplet
- https://huggingface.co/Omartificial-Intelligence-Space/Arabic-all-nli-triplet-Matryoshka
- https://huggingface.co/Omartificial-Intelligence-Space/Arabert-all-nli-triplet-Matryoshka
- https://huggingface.co/Omartificial-Intelligence-Space/Arabic-labse-Matryoshka
- https://huggingface.co/Omartificial-Intelligence-Space/Marbert-all-nli-triplet-Matryoshka
- https://huggingface.co/Omartificial-Intelligence-Space/Arabic-MiniLM-L12-v2-all-nli-triplet
Or the collection with all: https://huggingface.co/collections/Omartificial-Intelligence-Space/arabic-matryoshka-embedding-models-666f764d3b570f44d7f77d4e

My personal favourite is likely https://huggingface.co/Omartificial-Intelligence-Space/Arabert-all-nli-triplet-Matryoshka: a very efficient 135M parameters & scores #1 on https://huggingface.co/spaces/mteb/leaderboard.

urchade merterbak victor not-lain Omartificial-Intelligence-Space lunarflu osanseviero


I really like what the @jasperAITeam designed with Flash LoRA. It works really well for something that generates so quickly, and I'm excited to test it out with Animate Diff, because I recently was testing LCM on it's own for AD and the results were already promising.

I put together my own page of models using their code and LoRA. Enjoy!

https://huggingface.co/spaces/alvdansen/flash-lora-araminta-k-styles


Preview:
We will open source the 2.5B ChemVLM and the tool-enhanced ChemLLM-7B in the near future


nanoLLaVA-1.5 is here! Same size (1B), better performance 🔥🔥🔥
And it is much more powerful than v1.0
Try it out now on HF Spaces: https://huggingface.co/spaces/qnguyen3/nanoLLaVA
Model: https://huggingface.co/qnguyen3/nanoLLaVA-1.5

ZennyKenny YaTharThShaRma999 Blucky lunarflu mohamedemam osanseviero ucsahin appvoid qnguyen3 nicolay-r


Below we experiment with negative merger weighting (-1.0!) using task arithmetic. Merge formula on the model card and in the repo itself.

This model is steered to behave opposite to what MopeyMule demonstrated.

Based on the implications of the merge technique, we also propose Orthogonalized Vector Adaptation (OVA). We also extract a LoRA of the counter-refusal abliteration steering vector.

The resulting merger is not a perfect model, but it's a behaviorally interesting model. The model name was inspired by a Philip K. Dick story.
https://huggingface.co/grimjim/Llama-3-Perky-Pat-Instruct-8B

Refusal vector weights ready for use:
https://huggingface.co/grimjim/Llama-3-Instruct-abliteration-OVA-8B
https://huggingface.co/grimjim/Llama-3-Instruct-abliteration-LoRA-8B


An Open-source and super-fast alternative to @OpenAI  GPT4o is here! 🚀

In November last year, Iliad announced a fully open-source-oriented AI lab called @kyutai_labs  🧪

In this very short time they have released Moshi! An open speech-to-speech model 🗣... released publicly even before closed GPT4o (yes, you can try it right now!) 🌐

Demo: https://www.moshi.chat/?queue_id=talktomoshi

This is what you expect an intelligent companion to do! 🤖 It is continuously generating responses and listening at the same time with sub 300 ms latency! ⏱

This level of engagement is so new, it almost feels like I am under pressure to keep up with it! 😅

While it hallucinates like crazy! 🤯 I think fundamentally this is what a true assistant would look like. And did I say they are going to open-source it? 🆓

Weights and full technical report are promised to be coming soon! 📜

This is the work of an incredible but just 8 team members!! 👥👏


Qualitative Emergence: The Paradox of Statistical AI in Language Comprehension

https://empereur-pirate.medium.com/qualitative-emergence-the-paradox-of-statistical-ai-in-language-comprehension-ccd1e221423e

This text explores the nature of generative AI language models, highlighting their ability to produce coherent content despite being based on statistical processes. It discusses the role of random exploration in AI's impressive capabilities, while emphasizing that these models lack true understanding or subjectivity. The article contrasts AI's quantitative-to-qualitative approach with human cognitive development. It argues that AI serves as a tool for rewriting, analogical combination, and reasoned evaluation, rather than being truly intelligent. The text concludes by considering AI's potential applications in fields like law, stressing the importance of transparency and participatory models in AI development and use.


As we advance on the path towards true Artificial General Intelligence (AGI), it's crucial to recognize and address the limitations inherent in current technologies, particularly in large language models (LLMs) like those developed by OpenAI. While LLMs excel in processing and generating text, their capabilities are largely constrained to the domains of natural language understanding and generation. This poses significant limitations when dealing with more complex, abstract mathematical concepts such as topological analysis, 3D geometry, and homotopy type theory.

Topological Analysis and 3D Geometry: LLMs currently do not possess the inherent ability to understand or interpret the spatial and geometric data that is critical in fields like robotics, architecture, and advanced physics. These models lack the capacity to visualize or manipulate three-dimensional objects or comprehend the underlying properties that govern these forms.

Homotopy Type Theory is a branch of mathematics that combines homotopy theory and type theory. Homotopy type theory provides tools for a more robust handling of equivalences and transformations, something that LLMs are not designed to handle directly.

For the development of AGI, it is not sufficient to merely enhance existing models' capacities within their linguistic domains. Instead, a synthesis of symbolic AI with an understanding of homotopy type theory could pave the way. Symbolic AI, which manipulates symbols and performs logical operations, when combined with the abstract mathematical reasoning of homotopy type theory, could lead to breakthroughs in how machines understand and interact with the world.

To address these limitations we have developed Tenzin, which is a one-of-a-kind model with a planned release date within the next 1-2 weeks . To learn more join the waitlist at https://octave-x.com/.


New #NVIDIA paper: Improving Hyperparameter Optimization with Checkpointed Model Weights

Hyperparameter optimization often dominates the cost of model design.  So, we want cheap surrogate functions that approximate model performance to guide our search.  Existing methods can train on optimization metadata – like a trajectory of losses – to build these surrogates.

In our work, we add the ability to train our hyperparameter optimization surrogates on checkpointed model weights with a graph metanetwork.  This allows us to leverage a large, pre-existing source of information that can featurize the architecture, dataset, losses, and optimization procedure.

🔍Project page: https://research.nvidia.com/labs/toronto-ai/FMS/
👨💻 Code for reproduction: https://github.com/NVlabs/forecasting-model-search
📄 Full Paper: https://arxiv.org/abs/2406.18630

Our project was a collaboration between NVIDIA’s Toronto AI Lab and the TAO team.

Check out more work from Toronto AI Lab here: https://research.nvidia.com/labs/toronto-ai/

You can view the TAO toolkit here: https://developer.nvidia.com/tao-toolkit


The Universal Checkpointing paper is out! https://arxiv.org/abs/2406.18820

If you remember the Bigscience BLOOM-176B training, Tunji Ruwase and I co-invented this technology for Megatron-Deepspeed in order to enable to quickly scale up and down node topology while continuing training.

Since then the DeepSpeed team continued improving on that and it has now been fully integrated into Deepspeed.

The blog post is here: https://github.com/microsoft/DeepSpeed/blob/master/blogs/deepspeed-ucp/README.md


AI Agents Solved!
Not a drill and not lying, 100% success rate with GPT 3.5 and Swarm algorithms for AI Agents. GPT 3.5 will create and execute the Swarm algorithms, and they will complete the API call 100% of the time. Here is the Colab, play with it yourself. I have a Github Repo for it all too: https://colab.research.google.com/drive/1EF_JmPidwoCd8tEgOChUt6kCX7TBLe20?usp=sharing


I changed my mind about kaggle nb versions. They're goated


🌟 It's been about a week since @Google dropped Gemma 2 and now Gemma 2 27B is the highest-ranked open-source LLM on LMSYS Chatbot Arena Leaderboard, beating Meta Llama 3 70B and Alibaba Qwen 2 72B! 🚀💪

🔍 Here is what a week of Gemma looked like:

1⃣ First, it's a challenging model to run. The only reason I could find is soft-capping of logits within the attention for longer context optimizations. And no one was doing that before! 🤯

2⃣ Next, the technical report mentions a 2B model... where is it? 🤔

3⃣ Simple things like a context length of 8192 tokens, the Rotary Position Embeddings (RoPE), and the approximated GeGLU non-linearity are similar to earlier Gemma's. 📏🔄

4⃣ But a lot of new stuff is here like Local Sliding Window and Global Attention: they alternate between them at every layer... don't know why! 🤷♂ and of course Logit soft-capping. 💡

5⃣ On-Policy Distillation of Language Models - Knowledge Distillation: Leverage a larger teacher model to train a smaller model (the 9B model). 🎓➡📚

6⃣ Model Merging: Combined average models from experiments run with different hyperparameters. 🧪⚗

7⃣ And borrowed Grouped-Query Attention (GQA) from @Meta Llama-3. 🔄🤝

📖 One of the best articles on Gemma 2: https://huggingface.co/blog/gemma2

📊 Technical report: https://storage.googleapis.com/deepmind-media/gemma/gemma-2-report.pdf

🔗 Models: https://huggingface.co/collections/google/gemma-2-release-667d6600fd5220e7b967f315


Hello!

I've been in the lab, I think one or two of you saw my furtive attempts to create a dolphinized 2b Gemma, which is still waiting for more funding. I get paid in a week. `</3`

Once that funding ran out, I dropped my last pinch of API credits to work on this:

https://huggingface.co/datasets/DigitalClockwork/spatial_instruct_v1

It is an instruct database for spatial interactions with color tokens, i'm planning to tune a TBD model. Been experimenting with Gemma, but i'm welcome to ( smaller! ) model suggestions. If you think your favorite 0.5/0.75/1/2b can handle numbers, distances, or colors especially well, most especially community-enhanced models... I'm listening to the comments, intently!
Have a great day, and enjoy! This was one fun! 🤗

`-<3`


🐦⬛ Nexusflow/𝗡𝗲𝘅𝘂𝘀𝗥𝗮𝘃𝗲𝗻-𝗩𝟮-𝟭𝟯𝗕

🐦⬛ Nexusflow/𝗡𝗲𝘅𝘂𝘀𝗥𝗮𝘃𝗲𝗻-𝗩𝟮-𝟭𝟯𝗕

⌘ CohereForAI/𝗰𝟰𝗮𝗶-𝗰𝗼𝗺𝗺𝗮𝗻𝗱-𝗿-𝗽𝗹𝘂𝘀

⌘ CohereForAI/𝗰𝟰𝗮𝗶-𝗰𝗼𝗺𝗺𝗮𝗻𝗱-𝗿-𝗽𝗹𝘂𝘀

⛵ mistralai/𝗠𝗶𝘅𝘁𝗿𝗮𝗹-𝟴𝘅𝟮𝟮𝗕-𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁-𝘃𝟬.𝟭

⛵ mistralai/𝗠𝗶𝘅𝘁𝗿𝗮𝗹-𝟴𝘅𝟮𝟮𝗕-𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁-𝘃𝟬.𝟭

𝗬𝗼𝘂 𝗱𝗼𝗻'𝘁 𝗻𝗲𝗲𝗱 "𝗳𝘂𝗻𝗰𝘁𝗶𝗼𝗻 𝗰𝗮𝗹𝗹𝗶𝗻𝗴 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗶𝗻𝗴" 𝘁𝗼 𝗯𝘂𝗶𝗹𝗱 𝗴𝗼𝗼𝗱 𝗮𝗴𝗲𝗻𝘁𝘀 ⛔

It's trendy to share models "fine-tuned for function calling"; but from my observations, this fine-tuning is not necessary or sufficient to build good agent systems.
To name only a few:
🐦⬛ Nexusflow/𝗡𝗲𝘅𝘂𝘀𝗥𝗮𝘃𝗲𝗻-𝗩𝟮-𝟭𝟯𝗕
⌘ CohereForAI/𝗰𝟰𝗮𝗶-𝗰𝗼𝗺𝗺𝗮𝗻𝗱-𝗿-𝗽𝗹𝘂𝘀
⛵ mistralai/𝗠𝗶𝘅𝘁𝗿𝗮𝗹-𝟴𝘅𝟮𝟮𝗕-𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁-𝘃𝟬.𝟭
"Fine-tuned for function-calling" generally means "fine-tuned to generate function calls in correct JSON for extremely simple tasks". In other terms, it means "improve the formatting of the tool calls".

Yet I discovered two things while improving Transformers Agents:
🧐 Even when used as JSON agents, these fine-tuned models don't perform very well
🏅 𝙂𝙤𝙤𝙙 𝙗𝙖𝙨𝙚 𝙢𝙤𝙙𝙚𝙡𝙨 𝙥𝙚𝙧𝙛𝙤𝙧𝙢 𝙗𝙚𝙩𝙩𝙚𝙧 𝙬𝙞𝙩𝙝𝙤𝙪𝙩 𝙖𝙣𝙮 𝙛𝙞𝙣𝙚-𝙩𝙪𝙣𝙞𝙣𝙜, 𝙟𝙪𝙨𝙩 𝙥𝙡𝙖𝙞𝙣 𝙥𝙧𝙤𝙢𝙥𝙩𝙞𝙣𝙜. (Llama-3-70B-Instruct, GPT-4o, Claude-3.5-Sonnet)

👇 The graph below shows the count of errors for my GPT-4o validation run on the GAIA benchmark: 𝙰𝚐𝚎𝚗𝚝𝙿𝚊𝚛𝚜𝚒𝚗𝚐𝙴𝚛𝚛𝚘𝚛 and 𝙰𝚐𝚎𝚗𝚝𝙴𝚡𝚎𝚌𝚞𝚝𝚒𝚘𝚗𝙴𝚛𝚛𝚘𝚛 are the ones caused by incorrect formatting.
➤ As you can see, their count is already close to 0!
And given that GPT-4o is certainly not fine-tuned for our Code tool calling format, this shows that "function calling fine-tuning" is not necessary!

The hardest thing to get right in an agent is still to 𝙥𝙡𝙖𝙣 𝙜𝙤𝙤𝙙 𝙩𝙖𝙨𝙠-𝙨𝙤𝙡𝙫𝙞𝙣𝙜 𝙩𝙧𝙖𝙟𝙚𝙘𝙩𝙤𝙧𝙞𝙚𝙨 𝙤𝙫𝙚𝙧 𝙨𝙚𝙫𝙚𝙧𝙖𝙡 𝙨𝙩𝙚𝙥𝙨.
To improve this, we could:
- Use more powerful base models
- Make tool calling datasets with complex solving trajectories
- Use RL! cc @lvwerra

lvwerra merterbak AnkitAI louisbrulenaudet osanseviero Joseph717171 JackCloudman GPT007


X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic
  Textual Guidance (2303.15764)

Here's my cool outputs from "X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance" article. Their technique transforms mesh objects based on textual prompts. Featured here are a dynamically stylized 3D representation of Cristiano Ronaldo and a diamond dragon.

📄 paper:   https://huggingface.co/papers/2303.15764

osanseviero nikgr merterbak AIDANIAI AptAlbatross


🐦 Do you remember IBIS? Not a fancy bird but the open challenge in Inferring Binding Specificities of unexplored human transcription factors. Check our site (https://ibis.autosome.org/) and have a sip of fresh news below.

👥 More than 100 teams registered for the challenge yet only two dozen are using the opportunity to explore their models on the Leaderboard. Don't miss the chance to participate in the Leaderboard stage, although independently of that you can submit the final solution.

🌐 Remember, the training data for Leaderboard and Final are available online, and you are free to mix-and-match it in any combination.

🌌 For Leaderboard, we have received 650 total submissions of AAA (advanced ML) and 296 PWM models (a whopping set of 6682 PWMs in total).

🚀 For PWMs, the baseline is left far behind, but some TFs remain tough nuts to be cracked (see the attached figure 1).

📈 For AAAs, there is a solid improvement over the best-submitted PWMs in A2G, but the G2A discipline remains unpopular (see the attached figure 2). Free hint: this is your chance!

💡 Another free hint: If your model tends to overfit given a limited set of data for some TFs don't forget to use reverse-complement and shift augmentations. Also, don't hesitate to use multitarget models i.e. predicting the binding of multiple TFs at the same time.

💡 Last but not least, try to combine knowledge from all accessible experiment types, especially for G2A discipline (ChIP-Seq & genomic HT-SELEX) in a single model!

📣 Finally and importantly, following the requests from the community, we decided to EXTEND the Leaderboard until the final submission deadline.

🗓 The final submission deadline is also EXTENDED until Aug 15. The final submission form and details will be posted on the IBIS website in the first half of July, follow our Telegram group and mailing list (see the links at https://ibis.autosome.org).


Topo4D: Topology-Preserving Gaussian Splatting for High-Fidelity 4D Head
  Capture (2406.00440)

🔥🎭🌟 New Research Alert - ECCV 2024 (Avatars Collection)! 🌟🎭🔥
📄 Title: Topo4D: Topology-Preserving Gaussian Splatting for High-Fidelity 4D Head Capture 🔝

📝 Description: Topo4D is a novel method for automated, high-fidelity 4D head tracking that optimizes dynamic topological meshes and 8K texture maps from multi-view time-series images.

👥 Authors: @Dazz1e, Y. Cheng, @Ryan-sjtu, H. Jia, D. Xu, W. Zhu, Y. Yan

📅 Conference: ECCV, 29 Sep – 4 Oct, 2024 | Milano, Italy 🇮🇹

📄 Paper: https://huggingface.co/papers/2406.00440

🌐 Github Page: https://xuanchenli.github.io/Topo4D/
📁 Repository: https://github.com/XuanchenLi/Topo4D

🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers

🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers

🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #Topo4D #4DHead #3DModeling #4DCapture #FacialAnimation #ComputerGraphics #MachineLearning #HighFidelity #TextureMapping #DynamicMeshes #GaussianSplatting #VisualEffects #ECCV2024

DmitryRyumin Ryan-sjtu VanshRana12 prithivMLmods osanseviero AnkitAI


I've created a Stable Diffusion 3 (SD3) image generation space for convenience. Now you can:

1. Generate SD3 prompts from images
2. Enhance your text prompts (turn 1-2 words into full SD3 prompts)

https://huggingface.co/spaces/gokaygokay/SD3-with-VLM-and-Prompt-Enhancer

These features are based on my custom models:

- VLM captioner for prompt generation:
  - https://huggingface.co/gokaygokay/sd3-long-captioner

- Prompt Enhancers for SD3 Models:
  - https://huggingface.co/gokaygokay/Lamini-Prompt-Enchance-Long
  - https://huggingface.co/gokaygokay/Lamini-Prompt-Enchance

You can now simplify your SD3 workflow with these tools!

John6666 clem ucsahin osanseviero thliang01 AnkitAI netynet Best-codes Wok


MIRAI: Evaluating LLM Agents for Event Forecasting (2407.01231)

Check out our new benchmark paper on LLM agents for global events forecasting! https://huggingface.co/papers/2407.01231

📜 Arxiv: https://arxiv.org/abs/2407.01231
🔗 Project page: https://mirai-llm.github.io
💻 GitHub Repo: https://github.com/yecchen/MIRAI
📁 Dataset: https://drive.google.com/file/d/1xmSEHZ_wqtBu1AwLpJ8wCDYmT-jRpfrN/view?usp=sharing
📊 Interactive Demo Notebook: https://colab.research.google.com/drive/1QyqT35n6NbtPaNtqQ6A7ILG_GMeRgdnO?usp=sharing


LLMs are improving at math faster than my math coursework. I appreciate the all the hardworking engineers for helping me through high school.


✨🚀 Claude Sonnet 3.5 API.  It's already weaving digital magic!
🧠💻 Try it at my space: 🔗 https://huggingface.co/spaces/awacke1/AnthropicClaude3.5Sonnet-ACW

Kudos to @AnthropicAI for this elegant API! 👏 #AI #CodeMagic #AnthropicAI   Thanks Huggingface for hosting the best hub in the world for AI development!


DETRs Beat YOLOs on Real-time Object Detection (2304.08069)

🚀 Transformers are not here to take part but take over... and down goes real-time object detection! 💥

Enter Real-time DEtection Transformer (RT-DETR) 🦾 as suggested capable of real-time object detection. 🎯

Object DEtection Transformer (DETR) is not new (@Meta did it eons ago) but it had the issue of every other transformer, high computational cost 💸

RT-DETR brings an efficient hybrid encoder to expeditiously process multi-scale features by decoupling intra-scale interaction and cross-scale fusion to improve speed 🏎

Gist is RT-DETR speeds up object detection by redesigning its encoder to process features more efficiently and selecting higher quality initial object queries. ⚡

It also allows adjusting the number of decoder layers to balance speed and accuracy for different real-time scenarios. ⚖

This makes RT-DETR faster and more accurate than previous YOLO models. 🏆

How much better😎/faster? ⏱

RT-DETR-R50 achieved 53.1% AP on COCO and 108 FPS on a T4 GPU, while RT-DETR-R101 achieved 54.3% AP and 74 FPS, outperforming advanced YOLO models in both speed and accuracy. 🚀✨

📄 Paper: https://huggingface.co/papers/2304.08069

🧠 Models: https://huggingface.co/models?search=pekingu/rt-detr

merterbak clem osanseviero louisbrulenaudet


Just updated https://huggingface.co/spaces/MohamedRashad/timm-leaderboard with fuzzy search for people who want to search for a certian vision model


Search Hugging Face datasets by column names with a new experimental API! This API allows you to:

- Search for question-answering datasets that include context
- Find alpaca-style datasets
- Locate DPO datasets

Try it out here: https://huggingface.co/spaces/librarian-bots/dataset-column-search-api, or explore real-world applications in this notebook: https://huggingface.co/spaces/librarian-bots/dataset-column-search-api/blob/main/dataset_search_client_notebook.ipynb

clem osanseviero merterbak John6666 louisbrulenaudet


Real-time DEtection Transformer (RT-DETR) landed in transformers 🤩 with Apache 2.0 license 😍

🔖 models: https://huggingface.co/PekingU
🔖 demo: https://huggingface.co/spaces/merve/RT-DETR-tracking-coco
📝 paper: https://huggingface.co/papers/2304.08069
📖 notebook: https://github.com/merveenoyan/example_notebooks/blob/main/RT_DETR_Notebook.ipynb

YOLO models are known to be super fast for real-time computer vision, but they have a downside with being volatile to NMS 🥲

Transformer-based models on the other hand are computationally not as efficient 🥲

Isn't there something in between? Enter RT-DETR!

The authors combined CNN backbone, multi-stage hybrid decoder (combining convs and attn) with a transformer decoder. In the paper, authors also claim one can adjust speed by changing decoder layers without retraining altogether.
The authors find out that the model performs better in terms of speed and accuracy compared to the previous state-of-the-art. 🤩

merterbak hllj clem osanseviero multimodalart not-lain netynet sudzdpn louisbrulenaudet mathieu-chauvet Kabil007 Rbrq

talkative Tom-Neverwinter ooaykac GPT007


**How I train a LoRA: m3lt style training overview**

I've just written an article that takes a step by step approach to outlining the method that I used to train the 'm3lt' lora, a blended style model.

I've used the LoRA Ease trainer by @multimodalart  :D

https://huggingface.co/blog/alvdansen/training-lora-m3lt
https://huggingface.co/spaces/multimodalart/lora-ease

ijohn07 clem blender66cat not-lain thedarktrumpet ashemvets glpx IkeaMan stinkyyy

victor multimodalart rishiguin clem not-lain louisbrulenaudet


5,000 new repos (models, datasets, spaces) are created EVERY DAY on HF now. The community is amazing!

umuthopeyildirim osanseviero victor tail-call paulml jeremy-london brunatrevelin Clausss ZeroWw xi0v yuriachermann merterbak nroggendorff sted97 Jaromir not-lain netynet Heresynetwork SixOpen tarob0ba 3thn digiplay prithivMLmods GPT007 Nymbo

victor ngxson Clausss ZeroWw xi0v nold yuriachermann nroggendorff InferenceIllusionist sted97 danielus not-lain netynet Heresynetwork moock 3thn kramp GPT007 nampdn-ai IAmTheCollector

digiplay louisbrulenaudet GPT007 emirhanbilgic


🚨Exciting news for the Multilingual Synthetic Data Community!🚨

I’ve taken inspiration from the MAGPIE paper on Llama-3-8B-instruct and extended its capabilities. Here’s what’s new!

🗞 The MAGPIE paper showcased that if you use the instruction-tuned version (`Llama-3-8B-instruct`) to generate synthetic instructions and then fine-tune the base version (`Llama-3-8B`) on this dataset, you can improve even the it-tuned version

🤔 While reading a script by Sebastian Raschka, PhD, I wondered: Could these advancements be replicated in other languages? Specifically, could they benefit non-English datasets?

🎉 And the answer is YES! At least for Spanish. I've successfully adapted the techniques for Spanish, proving the model's flexibility and multilingual capabilities.

👩💻 To make this accessible, I created a basic script (heavily inspired by the Sebastian Raschka one) that allows you to generate similar datasets using `ollama` models (initially phi and llama3) automatically and upload it to the Hugging Face Hub!
[Script](https://gist.github.com/mrm8488/4650a5e3cc45523798a527a3446eb312)

🔍 Explore the datasets 📚 generated using our new script!

- [Llama-3-8B](https://huggingface.co/datasets/mrm8488/dataset_llama3_5000_samples_es_4231_filtered)
- [Phi-3-medium](https://huggingface.co/datasets/mrm8488/dataset_phi3-medium_5000_samples_es_3906_filtered)
-  [Phi-3-mini](https://huggingface.co/datasets/mrm8488/dataset_phi3_5000_samples_es_3282_filtered)

Note: These datasets have basic filtering. Apply additional quality filters before using them to fine-tune large language models.

Inspiration and base script:
https://github.com/rasbt/LLMs-from-scratch/blob/main/ch07/05_dataset-generation/llama3-ollama.ipynb
https://www.linkedin.com/feed/update/urn:li:activity:7210982019751661568/

Davipar osanseviero apol anakin87 davanstrien victor ucsahin floschne GPT007 vikas Marvin73 pavaldeveloper mrm8488 Taylor658 adorkin


I have finished writing a blogpost about building an image-based retrieval system,  This is one of the first-ever approaches to building such a pipeline using only open-source models/libraries 🤗

You can checkout the blogpost in https://huggingface.co/blog/not-lain/image-retriever and the associated space at https://huggingface.co/spaces/not-lain/image-retriever .

✨ If you want to request another blog post consider letting me know down below or you can reach out to me through any of my social media

📖 Happy reading !

tail-call YaTharThShaRma999 umutphp John6666 MrOvkill sted97 kleberbaum vincentweisser BigOne


Yet another rewarding week in Open Source AI:

1. Google dropped Gemma 27B & 9B - The best open (commercially permissive) LLM out there, according to LYMSYS.
https://huggingface.co/collections/google/gemma-2-release-667d6600fd5220e7b967f315

2. Mars5 TTS - Text to Speech with insane prosodies control & voice cloning.
https://huggingface.co/CAMB-AI/MARS5-TTS

3. Meta shipped LLM Compiler - beats GPT 4 on code optimisation and compiler reasoning.
https://huggingface.co/collections/facebook/llm-compiler-667c5b05557fe99a9edd25cb

4. Arcee-Spark - Qwen2 7B (w/ merging) fine-tuned further to beat GPT 3.5 on MT Bench.
https://huggingface.co/arcee-ai/Arcee-Spark

5. Gemini Nano out in the wild in Chrome - On device LLM with just 2 lines of code (fully offline)

6. Fal released a fully Open Source GAN based Super-Resolution model (with second version already cooking)
https://huggingface.co/fal/AuraSR

7. NYU release Cambrian 1 - Vision Multimodal LLM that beats pretty much all other closed source competition 8-34B model size
https://huggingface.co/nyu-visionx

And.. much more like Open LLM Leaderboard got a major update, LYMSYS released Chat Vision Arena, OpenAI released a paper on CriticGPT!

What a lovely week, can’t wait for the next to see what the community is up to! Put it down in comments if I missed something 🔥

julien-c netynet m-ric jeffboudier SixOpen hsnugavin brandonisntafraid pcuenq louisbrulenaudet victor polinaeterna apolloparty clem tomaarsen ogimgio qnixsynapse linoyts John6666 Nelathan ruben-wleon ddiddi Joseph717171

dashfunnydashdash ddiddi Joseph717171 bradshimmin


Understanding and Diagnosing Deep Reinforcement Learning (2406.16979)

The paper is now available at ICML 2024. This paper introduces a foundational approach to analyze deep reinforcement learning decision making. Truly excited to share these results!

Paper: https://openreview.net/pdf?id=s9RKqT7jVM
HF: https://huggingface.co/papers/2406.16979
Twitter: https://x.com/EzgiKorkmazAI/status/1798990328390996002


Announcing the creation of the "HF for Legal" organization, an open-source community dedicated to demystifying language models for legal professionals 🤗

Whether you're a practicing attorney, a legal scholar, or a technologist interested in legal applications of AI, HF for Legal may be your hub for exploration, learning, and free innovation ⚗

On the occasion of this launch, you'll be able to find several notebooks I've been developing over the last few months for TSDAE pre-training of embedding models, the generation of indexes for semantic search, based on the formidable work of @tomaarsen and @nreimers, adapted to the field of French law, or the addition of information retrieval tasks to the MTEB.

Join us in our mission to make AI more accessible and understandable for the legal world, ensuring that the power of language models can be harnessed effectively and ethically.

Link to the org: https://huggingface.co/HFforLegal

Special thanks to @clem for encouraging me to start this organization. Let's hope we can bring together all the enthusiasts who work in this field.

Let's code and share together! 🚀🔗

tomaarsen clem not-lain netynet GFA-D2 ucalyptus Joseph717171 Elizezen jambracken vincentweisser ruben-wleon TuringsSolutions nroggendorff


𝐖𝐡𝐚𝐭 𝐬𝐩𝐚𝐫𝐤𝐞𝐝 𝐭𝐡𝐢𝐬 𝐢𝐝𝐞𝐚?

𝐖𝐡𝐚𝐭 𝐬𝐩𝐚𝐫𝐤𝐞𝐝 𝐭𝐡𝐢𝐬 𝐢𝐝𝐞𝐚?

𝐇𝐨𝐰 𝐝𝐢𝐝 𝐈 𝐜𝐫𝐞𝐚𝐭𝐞 𝐲𝐨-𝐋𝐥𝐚𝐦𝐚?

𝐇𝐨𝐰 𝐝𝐢𝐝 𝐈 𝐜𝐫𝐞𝐚𝐭𝐞 𝐲𝐨-𝐋𝐥𝐚𝐦𝐚?

How to alter the behavior of a Language Model without fine-tuning or prompting? Say hello to 🎤 yo-Llama 🦙!

Model https://huggingface.co/anakin87/yo-Llama-3-8B-Instruct

This experiment steers Llama-3-8B-Instruct to respond in a rap style.
How? Amplifying the rap direction in the activation space. 😎

𝐖𝐡𝐚𝐭 𝐬𝐩𝐚𝐫𝐤𝐞𝐝 𝐭𝐡𝐢𝐬 𝐢𝐝𝐞𝐚?

Lately, I got interested in mechanistic interpretability of LLMs.

💡 A recent paper, "Refusal in Language Models Is Mediated by a Single Direction," showed how to find the refusal direction in the activation space of Chat Language Models and either erase or amplify it.
A clever jailbreak method for open weights models.

Then, @failspy took it a step further by modifying the models to amplify different traits, such as making a model seem grumpy or irritable.

𝐇𝐨𝐰 𝐝𝐢𝐝 𝐈 𝐜𝐫𝐞𝐚𝐭𝐞 𝐲𝐨-𝐋𝐥𝐚𝐦𝐚?
(📓 notebook in the HF repository, heavily inspired by Failspy's work)

1⃣ Load the Llama-3-8B-Instruct model.
2⃣ Load 1024 examples from Alpaca (instruction dataset).
3⃣ Prepare a system prompt to make the original model act like a rapper.
4⃣ Run inference on the examples, with and without the system prompt, and cache the activations.
5⃣ Compute the rap feature directions (one for each layer) from the activations.
6⃣ Apply the feature directions one by one, checking the results on some examples.
7⃣ Pick the best-performing feature direction.
8⃣ Apply this feature direction and voilà!
yo-Llama-3-8B-Instruct is born! 🥳🎶

This was a fun experiment.

📚 Resources

Refusal in Language Models Is Mediated by a Single Direction - https://arxiv.org/abs/2406.11717

Uncensor any LLM with abliteration: great practical blog post by @mlabonne  https://huggingface.co/blog/mlabonne/abliteration

Practical materials by @failspy
- abliterator library https://github.com/FailSpy/abliterator
- Llama-MopeyMule-3-8B-Instruct model (+ notebook) https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule

mlabonne diwank YaTharThShaRma999 Korakoe


🚀 Llama-3-ELYZA-JP-8B

ELYZA, Inc. has developed two large language models (LLMs) for Japanese called "Llama-3-ELYZA-JP-70B" with 70 billion parameters and "Llama-3-ELYZA-JP-8B" with 8 billion parameters, based on Meta's "Llama 3" series. These models have been fine-tuned through additional pre-training and post-training to improve Japanese language capabilities significantly.

Key Points:

Performance:
- Llama-3-ELYZA-JP-70B surpasses global models such as GPT-4, Claude 3 Sonnet, and Gemini 1.5 Flash.
- Llama-3-ELYZA-JP-8B matches models like GPT-3.5 Turbo and Claude 3 Haiku despite having fewer parameters.

Availability:
- The 8B model is available on Hugging Face Hub and can be used for both research and commercial purposes under the Llama 3 Community License.

Methodology:
- ELYZA enhanced the Japanese performance of the Llama 3 models through additional training with high-quality Japanese corpora and Instruction Tuning with proprietary datasets.

Benchmarks:
- Evaluations using ELYZA Tasks 100 and Japanese MT-Bench showed significant improvements in Japanese language generation.

Inference Speed:
- To address inference speed issues due to model size, ELYZA implemented Speculative Decoding, which achieved up to 1.6 times faster inference for the 70B model.

Overall, ELYZA's models demonstrate state-of-the-art performance in Japanese language tasks and are optimized for both efficiency and effectiveness.

Model URL:
- https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B
- https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B-AWQ
- https://huggingface.co/elyza/Llama-3-ELYZA-JP-8B-GGUF

Blog post (in Japanese):
https://note.com/elyza/n/n360b6084fdbd


📢 I've tested google/Gemma-2-9b-it in Target Sentiment Analysis (TSA), in zero-shot learning mode on RuSentNE-2023 dataset with texts translated into English (🇺🇸).

🔎 Findings: The key contribution with the most recent Gemma-2 release is reasoning alignment between different langauges. This is basically the first model under 10B category which shows equal results in English and non-english texts. In the case of texts in English it performs similar to LLaMa-3-8B / Mistal-7B

Model: https://huggingface.co/google/gemma-2-9b-it
Benchmark: https://github.com/nicolay-r/RuSentNE-LLM-Benchmark


On-Policy Distillation of Language Models: Learning from Self-Generated
  Mistakes (2306.13649)

📅 Remember when at the beginning of the year @Google  gave an update on knowledge distillation! Introducing a way of learning from Self-Generated Mistakes?

📊 Resulting in significant improvements across tasks:
- 📄 2.1x in summarization
- 🌐 1.7x in translation
- 🧠 1.9x in reasoning tasks

🚀 Well, it looks like Google wasn't messing around! According to the Gemma 2 tech report, knowledge distillation was used to pre-train the 9B model, while the 27B model was pre-trained from scratch.

📈 For post-training, the Gemma 2 team generated completions from a stronger teacher model (unspecified in the report, but presumably Gemini Ultra), and then trained the student models on this synthetic data with SFT. This is quite common as seen in many open models, such as Zephyr and OpenHermes.

🤔 Sounds too good to be true? These models suffer from distribution mismatch between output sequences seen during training and those generated by the student during inference.

📰 This is where the January 2024 paper "On-Policy Distillation of Language Models" comes in...

🔍 Gemma 2 team used “on-policy distillation,” where the student generates completions from the SFT prompts. These completions are then used to compute the KL divergence between the teacher’s and student’s logits. By minimizing the KL divergence throughout training, the student learns to model the behavior of the teacher accurately while also minimizing the train-inference mismatch.

📚 Gem🔹 of a blog by @huggingface  uncovering everything Gemma 2: https://huggingface.co/blog/gemma2#knowledge-distillation

📄 On-Policy Distillation of Language Models: https://huggingface.co/papers/2306.13649