Want to read curated list of papers by @akhaliq in your mail box?

Thanks to the API provided by Hugging Face, I made a simple GitHub Action based newsletter bot to send out 🤗 Daily Papers. Check out the attached video clip to get a sense of what it is!

Internally, it leverages Gemini API to assign tags for each paper, and all papers are archived by tags and batches. Of course, you can directly go to the papers' pages from your mail box to check out the full paper!

Since everything is automated, GitHub Action and Gemini API are free, and the subscription management is free via Google Groups, this newsletter bot is entirely free. Furthermore, if you wish, you could fork the project for your own newsletter service.

subscription: https://groups.google.com/g/hf-daily-paper-newsletter
project repo: https://github.com/deep-diver/hf-daily-paper-newsletter

In the next step, I will experimentally add auto translation (to Korean) feature for every papers.

chansung Dlbk lbourdois victor osanseviero julien-c Tonic akhaliq pierrci clem mishig merve dashfunnydashdash Moguiy AdinaY abidlabs rwightman veerpareek samusenps ysharma nbroad inmatrix Oliver-Guy

osanseviero julien-c chansung Tonic akhaliq clem linoyts AdinaY mariagrandury samusenps ysharma


🙋🏻♂Hey there folks ,

i wanted to share with you a really cool new organisation called

https://huggingface.co/lowres

In just one week it has gathered almost 150 members !

Check them out if you love anime , SDLX, LORAs and cool datasets.

can we make this one reach 200 members? 🚀

not-lain julien-c clem merve s3nh akashicmarga AdinaY radames samusenps ppaon sbrandeis


🤦🏻♂well, day before yesterday i was so happy about **gpuzero** that i made a bunch of demos : https://huggingface.co/posts/Tonic/802671427380916

- one for YI-200K , but it actually doesnt quite fit on a GPUZero... U_U
- one for SDXL style align, but omg i didnt even realize at the time it wasnt my demo of it (lol)
- one for texify (which works great btw, keep an eye on texify, it's about to blow up... in a couple of months!)

so yeah, i ran back and tried to get my demos working at least for sdxl which i love , but i simply couldnt get the CPU stuff working, or the refactored code working. no wonder i was thinking "wow this is so easy" on @osanseviero 's demo : yeah , it's not my code that's why it works 😅🙏🏻

anyway spent the day unsuccessfully experimenting, but starting tomorrow i'll try to serve some cool and overlooked models so 🤗huggingface appreciators can try them out 🚀

osanseviero hogiahien victor clem julien-c jarvisx17 Solshine


Shocking: 2/3 of LLMs fail at 2K context length

code_your_own_ai makes a great vlog about mostly LLM related AI content.
As I watched the video below, I wondered about current best practices on LLM evaluation. We have benchmarks, we have sota LLMs evaluating LLMs, we have tools evaluating based on human comparison.
Often, I hear, just play with the LLM for 15 mins to form an opinion.
While I think for a specific use case and clear expectations, this could yield signal carrying experiences, I also see that one prompt is used to judge models.
While benchmarks have their weaknesses, and are by themselves not enough to judge model quality, I still think systematic methods that try to reduce various scientifically known errs should be the way forward, even for qualitative estimates.
What do you think? How can we make a public tool for judging models like lmsys/chatbot-arena-leaderboard help to leverage standards known in social science?

https://www.youtube.com/watch?v=mWrivekFZMM

clem osanseviero julien-c pierrci NameeO radames AdinaY Tanvir1337 ucalyptus frontenbrecher


Here is my selection of papers for today (12 Jan)

https://huggingface.co/papers

PALP: Prompt Aligned Personalization of Text-to-Image Models

Object-Centric Diffusion for Efficient Video Editing

TRIPS: Trilinear Point Splatting for Real-Time Radiance Field Rendering

Diffusion Priors for Dynamic View Synthesis from Monocular Videos

Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation

TOFU: A Task of Fictitious Unlearning for LLMs

Patchscope: A Unifying Framework for Inspecting Hidden Representations of Language Models

Secrets of RLHF in Large Language Models Part II: Reward Modeling

LEGO:Language Enhanced Multi-modal Grounding Model

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

Tuning LLMs with Contrastive Alignment Instructions for Machine Translation in Unseen, Low-resource Languages

A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism

Towards Conversational Diagnostic AI

Transformers are Multi-State RNNs

Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training

Distilling Vision-Language Models on Millions of Videos

Efficient LLM inference solution on Intel GPU

TrustLLM: Trustworthiness in Large Language Models

clem osanseviero namtran victor chansung ronzyponz davanstrien Tonic sackoh Dlbk samusenps AdinaY alielfilali01 iarata


Let's play a little game, how would you build the Rabbit R1 with open source tech? Here is my stack:

  - https://huggingface.co/openai/whisper-small for awesome Speech-to-Text with low latency
  - https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1 for an awesome super powerful LLM Brain
  - https://huggingface.co/coqui/XTTS-v2 for a nice and clean voice

Which stack will you personally choose?

kramp osanseviero mwz mohitsha abhishek not-lain clem jahnviatls n4ze3m ManOhMan taufiqdp Sandeepa KnutJaegersberg eek lunarflu jonChengRX0 RalphX1 m-ric timqian rbiswasfc julien-c mrfakename marcsun13 radames johannhartmann Halfshadow Dlbk ucalyptus mishig Moguiy luizgsbraz 10MinuteMan lulzx jesusoctavioas cashion

merve clem yezzer vedalken VictorSanh KnutJaegersberg auditor lunarflu BrigitteTousi RalphX1 loubnabnl julien-c hensam92 mrfakename radames Dlbk mishig clefourrier sosoai SvCy Docfile neiltron 10MinuteMan behlock sbrandeis

clem RalphX1 julien-c pierrci mrfakename


Sharing a super-fast segmentation model today 💨
SlimSAM is pruned-distilled version of SAM model, it's (up-to 8.6x) faster and smaller yet very powerful! ⚡
It has the same architecture as SAM, meaning you can use the 🤗 transformers code for SAM on SlimSAM models ⬇  (yes only 3 lines of code!)
```python
from transformers import pipeline
generator = pipeline(model="nielsr/slimsam-50-uniform", task="mask-generation")
outputs = generator(image)
```
Lastly, I have built an app for you to compare SlimSAM and SAM outputs
https://huggingface.co/spaces/merve/slimsam

osanseviero linoyts abhishek clem lunarflu BrigitteTousi marcsun13 julien-c Pclanglais Dlbk samusenps Tonic sbarman25 pcuenq afrideva sbrandeis


As a GPU poor, I found a nice open source project.

dstackai is the perfect open source project for GPU poor. Simply specify resource requirements (GPU RAM, spot, ...), then will suggest the cheapest options among the popular GPU cloud providers (AWS, GCP, Azure, Lambda Labs, TensorDock, and vast.ai)

Provision VM instances in 3 different use cases. These are the essential for any ML projects.
- Dev: connect provisioned VM instance to your fav IDE (including Jupyter)
- Task: run experiments (training, fine-tuning, ...) via SSH
- Service: run your model in production via HTTPS

dstack is 100% open source, but you need to have your own accounts for each GPU cloud provider, enough GPU quota, configure credentials, etc., all by yourself. Luckily, dstack will announce dstack cloud which let you not worried about all the hassles. The price is almost as same as you directly connect to each cloud with your account.

The attached code snippet shows you how to provision a Mistral-7B model in Text Generation Inference(TGI) on the cheapest VM instance (of having 24GB of VRAM). Then you get the HTTPS connection to it, and play with it as usual with TGI client library as attached in the second code snippet.

If you want to learn more about dstack, check out the official website. Without GPU sponsors, as an individual open source contributor in ML, this kind of project is pretty important.
: https://dstack.ai/

If you are looking for an alternative, there is SkyPilot project as well
: https://github.com/skypilot-org/skypilot

peterschmidt85 chansung osanseviero lysandre merve victor sayakpaul linoyts abhishek abidlabs KnutJaegersberg auditor johannhartmann julien-c radames bitdeep samusenps mishig gblazex juyongjiang greencookies akhaliq Citaman sbrandeis

peterschmidt85 lysandre merve KnutJaegersberg julien-c pierrci samusenps juyongjiang akhaliq


https://huggingface.co/fblgit/UNA-dolphin-2.6-mistral-7b-dpo-laser

RE-Introducing, some of the best SFT model, he legend: DOLPHIN.  This model is very special,  a LASER-UNA model: UNA-dolphin-2.6-mistral-7b-dpo-laser

@fblgit in collaboration with @fernandofernandes and @ehartford

fblgit osanseviero den0620 radames lysandre abhishek victor m-ric julien-c dvilasuero tshrjn robertpiosik pacman2473 AtAndDev PatrickZhang16


This is my first post! Thanks to @victor for adding me!

lysandre victor sayakpaul abhishek clem julien-c Dlbk


ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of
  Video (2401.05314)

Score Distillation Sampling with Learned Manifold Corrective (2401.05293)

InseRF: Text-Driven Generative Object Insertion in Neural 3D Scenes (2401.05335)

PIXART-δ: Fast and Controllable Image Generation with Latent
  Consistency Models (2401.05252)

URHand: Universal Relightable Hands (2401.05334)

The Impact of Reasoning Step Length on Large Language Models (2401.04925)

Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk (2401.05033)

Here is my selection of papers for today (11 Jan)

https://huggingface.co/papers

ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of Video https://huggingface.co/papers/2401.05314

Score Distillation Sampling with Learned Manifold Corrective https://huggingface.co/papers/2401.05293

InseRF: Text-Driven Generative Object Insertion in Neural 3D Scenes https://huggingface.co/papers/2401.05335

PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Models https://huggingface.co/papers/2401.05252

URHand: Universal Relightable Hands https://huggingface.co/papers/2401.05334

The Impact of Reasoning Step Length on Large Language Models https://huggingface.co/papers/2401.04925

Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk https://huggingface.co/papers/2401.05033


 🙋🏻♂hey there folks , 🌟Tonic here
- just a 🛠builder from 🗼Paris !
Everyone is making something special for their first post , so since i got access to **GPUZero** , well, my first post is about **GPUZero**

### GPUZero is here !

This one's great for builders like me that are often making and serving models to their community.
- demos get popular then fade away
- they retain interest over the next three months as folks have questions

**GPUZero** lets you serve demos to your community over time while optimizing for costs .
Believe it or not it's actually impossible to pay for everything over a whole month if you have even one GPU running at a time.
I'm so excited for this because it lets  me serve a complete stack of specialized models and to build with them too.
- all optimized for efficiency in dollar cost.

check out some demos that are available on GPUZero :
- https://huggingface.co/spaces/Tonic/marker-texify : this one  is the first one i made it's for an image to latex formula model.
-  https://huggingface.co/spaces/Tonic/YI-6B-200k : this one probably actually works better on GPUZero than on a standard A10, but dont take my word for it , try it out 🤗
- https://huggingface.co/spaces/Tonic/style-aligned_sdxl : this one was my greatest technical achievement, check the dates and times on it too, there's a backstory to this one so i'll maybe tell it in another post

osanseviero mlabonne lunarflu hogiahien merve victor julien-c radames bigbeachdaddy pcuenq samusenps Chunte sharad

osanseviero lunarflu merve mrfakename julien-c bigbeachdaddy pcuenq


I finished my model merging experiment day.🤗I would love your thoughts on this.

What did I do? I merged Mistral Instruct 0.1 and 0.2 models using different merging techniques:
- SLERP: linear interpolation (most popular method)
- MoE: replace some forward layers with MoE layers; using a random gate for now
- Frankenmerge: also known as passthrough, but that isn't very cool. It concatenates some specified layers ending in different numbers of params. In my case, I went from 7B to 9B.

Note: merging is not building an ensemble of models. You can read more about merging techniques at https://huggingface.co/blog/mlabonne/merge-models

Results
I built the 3 models using mergekit (running in an HF Space) - took less than an hour to do the three) https://huggingface.co/collections/osanseviero/mistral-instruct-merges-659ebf35ca0781acdb86bb0a

I'm doing a quick check with the OpenLLM Leaderboard.
🚨The OpenLLM Leaderboard is more suitable for pre-trained models than instruct models, but I still thought it would be interesting to look at the insights🚨

You can look at the attached image. Some interesting things
- All three models performed somewhere between 0.1 and 0.2 - congrats to the 140 people who got it right in https://twitter.com/osanseviero/status/1745071548866736171
- Frankenmerge terribly sucked with GSM8K. It seems that adding some Mistral 0.1 layers actually degraded the performance a lot  - this is worse than even 0.1!
- Otherwise, frankenmerge was decent across HellaSwag, MMLU, and specially TruthfulQA
- MoE is using random gating, so I expected something right in between 0.1 and 0.2, which was the case

What do I do with this?
Not sure tbh! I think doing proper MT bench evals would be nice. I also think all of us should give a nice GH star to mergekit because it's awesome. I would love to have the time to do end-to-end ablation studies, but cool new things are coming up. Let me know if you have any thoughts in the results

davanstrien merve mlabonne mrm8488 abhibisht89 gblazex eitanturok detakarang dvilasuero victor julien-c kgourgou thomwolf mwitiderrick not-lain pcuenq marcsun13 clem chansung radames lysandre KnutJaegersberg toshas Mohammadreza m-ric ernestp56 wenqiglantz bhadresh-savani Dlbk nouamanetazi samusenps Csplk Warung jiandong chrissarmstrong sbrandeis Haleshot

davanstrien pprp dvilasuero julien-c not-lain mrfakename clem chansung lysandre KnutJaegersberg DrishtiSharma nouamanetazi macadeliccc

mrm8488 julien-c not-lain clem chansung Haleshot


Last month was great for faster/smaller segmentation models, and I wanted to dedicate my first post to compile the recently released SAM variants! 🤗
📚 All models and their demos can be found in this collection 👉🏼 https://huggingface.co/collections/merve/segment-anything-model-6585835fc76915aa14e2bcbd
The ideas behind them are mostly about making heavy image encoder lighter either through distillation or changing the pre-training. 💡
⚡MobileSAM: It decouples the heavy image encoder of SAM and distills it into a TinyViT to make SAM smaller. The architecture is same except for the encoder.
⚡TinySAM: It distills the whole model with online hard prompt sampling. The authors also quantized it and released Q-TinySAM.
⚡ EfficientSAM: This model combines masked image pre-training for training lightweight image encoders (like ViTMAE, learns to reconstruct the images) and mask decoder.
⚡ FastSAM: It's a CNN-based model where the problem is modeled as segments generation. The inference takes place as everything is segmented at once and then you can prompt with boxes or points or text (and this is how it is similar to SAM). So the architecture is nowhere similar to original SAM itself.
✨ [NEW] SlimSAM: It's a pruned-distilled version of pre-trained SAM. The architecture is same so @nielsr recently converted the weights and you can use it with the same API you use with SAM models. You can find the available checkpoints in the collection.
I hope you liked it!

osanseviero lopezjhonf NemesisAlm amyeroberts julien-c mattmdjaga KnutJaegersberg bisnotforbella AdinaY samusenps abidlabs

Norod78 julien-c KnutJaegersberg mlabonne AdinaY sbrandeis


🔥 Less is more for DPO, high quality matters!

📢 Dropping our first open dataset and LLM of the year:

💾Meet distilabel Orca Pairs DPO, an improved version of the now famous dataset from Intel:

https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs

🏛 And a new OpenHermes fine-tune outperforming baselines with 54% less DPO pairs:

https://huggingface.co/argilla/distilabeled-Hermes-2.5-Mistral-7B

You can use this new dataset for your DPO tuning, just like this:

```
from datasets import load_dataset

# Instead of this:
# dataset = load_dataset("Intel/orca_dpo_pairs", split="train")

# use this:
dataset = load_dataset("argilla/distilabel-intel-orca-dpo-pairs", split="train")

dataset = dataset.filter(
    lambda r:
        r["status"] != "tie" and
        r["chosen_score"] >= 8 and
        not r["in_gsm8k_train"]
)
```
This will reduce the size of the original by 54% while giving you better quality preferences!

What should we build next?

davanstrien osanseviero merve radames tomaarsen mlabonne alielfilali01 clem linoyts KnutJaegersberg Forbu14 lysandre d0rj mammour VictorSanh gblazex sethuiyer eliebak sbrandeis

davanstrien osanseviero merve radames tomaarsen BramVanroy clem KnutJaegersberg lixin67


Finally found my go-to hat for 2024 😎

Thanks to https://huggingface.co/fal-ai

dvilasuero burkaygur radames merve pierrci not-lain clem clefourrier samusenps MarinaraSpaghetti fedyanin dillfrescott roseking sbrandeis

lysandre merve clem roseking dillfrescott


Here is my selection of papers for today (10 Jan)

https://huggingface.co/papers

Jump Cut Smoothing for Talking Heads

FADI-AEC: Fast Score Based Diffusion Model Guided by Far-end Signal for Acoustic Echo Cancellation

Masked Audio Generation using a Single Non-Autoregressive Transformer

Let's Go Shopping (LGS) -- Web-Scale Image-Text Dataset for Visual Concept Understanding

Narrowing the Knowledge Evaluation Gap: Open-Domain Question Answering with Multi-Granularity Answers

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding

MagicVideo-V2: Multi-Stage High-Aesthetic Video Generation

not-lain pierrci julien-c abidlabs davanstrien osanseviero

not-lain julien-c abidlabs davanstrien alielfilali01


hello world!
we're starting a new recurring event/club where we read and implement cool  ai papers on skunkworks discord. first paper we chose is self-play as there are a lot of opportunities to expand on this framework, here's the link for the event: https://discord.gg/eAgBr7Fy?event=1194392774905172030

im plannin my next post to be a technical deepdive of PCN and ProspectiveConfiguration algo as ive been spending the last few days getting a good grasp at this promising alternative to BP, stay tuned.

osanseviero Tonic victor pierrci pcuenq merve samusenps sbrandeis


Here is my selection of papers for today (9 Jan)

https://huggingface.co/papers

AGG: Amortized Generative 3D Gaussians for Single Image to 3D

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

DiarizationLM: Speaker Diarization Post-Processing with Large Language Models

TeleChat Technical Report

Soaring from 4K to 400K: Extending LLM's Context with Activation Beacon

AST-T5: Structure-Aware Pretraining for Code Generation and Understanding

Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach

Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM

GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation

CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution

Mixtral of Experts

mwz alckasoc Tonic victor merve davanstrien IlyasMoutawwakil lixin67 aust-t


Full fine-tuning of Microsoft's Phi2 on a single 4090 is now supported in axolotl. Thanks to @abacaj and @vikhyatk for their help with gradient checkpointing and flash attention fixes.

alpaca finetune: https://huggingface.co/openaccess-ai-collective/phi2-alpaca
wandb: https://wandb.ai/oaaic/phi2/runs/00pc4ugb?workspace=user-wing-lian
merged PR: https://github.com/OpenAccess-AI-Collective/axolotl/pull/1058

osanseviero Citaman victor kramp Tonic marcsun13 merve ybelkada Zmu samusenps mrfakename Chunte JaiSurya radames

osanseviero Tonic merve ybelkada Chunte radames SicariusSicariiStuff sbrandeis


Here is my selection of papers for today (8 Jan)

https://huggingface.co/papers

DocGraphLM: Documental Graph Language Model for Information Extraction

Denoising Vision Transformers

Progressive Knowledge Distillation Of Stable Diffusion XL Using Layer Level Loss

Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively

Pheme: Efficient and Conversational Speech Generation

DeepSeek LLM: Scaling Open-Source Language Models with Longtermism

Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache


QuIP# ecosystem is growing :)

I've seen a quip# 2 bit Qwen-72b-Chat model today on the hub that shows there is support for vLLM inference.
This will speed up inference and make high performing 2 bit models more practical. I'm considering quipping MoMo now, as I can only use brief context window of Qwen-72b on my system otherwise, even with bnb double quantization.

https://huggingface.co/keyfan/Qwen-72B-Chat-2bit

Also notice the easier to use Quip# for all library :)

https://github.com/chu-tianxiang/QuIP-for-all

Felladrin osanseviero victor ybelkada reach-vb merve samusenps mexicanamerican

osanseviero ybelkada reach-vb Tonic kramp merve abidlabs

osanseviero ybelkada reach-vb merve samusenps sbrandeis


👋 Hi there!

This is my very first post.

I'll use  it to share some old news: a math preference dataset for DPO!

I created this dataset some time ago while we were developing distilabel (https://github.com/argilla-io/distilabel).

Some days ago we found out people are actually using it! So I'll use this post to explain how I built it in case it's useful for the community.

1. I used distilabel's SelfInstruct-inspired task to generate instructions about different math topics. I curated the instructions with Argilla (on Spaces!).
2. Then I used a distilabel Pipeline to build a preference dataset using gpt3.5 as generator and gpt4 as labeller. If I recall correctly I used our JudgeLM implementation (see https://distilabel.argilla.io/latest/technical-reference/tasks/#judgelmtask)

(see the screenshot with the dataset in the Argilla UI)

3. Then I just binarized into chosen, rejected pairs and voilà:

https://huggingface.co/datasets/argilla/distilabel-math-preference-dpo

The funny thing is that I used this to do a second DPO run over Notus-7B. I hoped to see an improvement on math/reasoning skills but it actually improved in STEM and Humanities and did worse on Math 🤣 .

In conclusion, this dataset was only a quick experiement. I'm happy to see the community found it useful. Data for DPO and fine-tuning are still a mystery, let's unveil these mysteries in 2024 together!

Follow me for the most exciting datasets for LLMs (and maybe some great, small, efficient models). I plan to announce all Argilla open-source work here!

pacoid sparkycollier osanseviero gabrielmbmb mishig severo sugatoray davanstrien mlabonne qJakc MoritzLaurer KnutJaegersberg radames victor clem julien-c merve tmnam20 Erland DarkLord7771 abidlabs Masa-Erland

pcuenq osanseviero gabrielmbmb mishig plaguss davanstrien pierrci qJakc thomwolf Manel-Hik victor radames mariagrandury clem julien-c merve sbrandeis

qJakc Tonic mariagrandury julien-c davanstrien merve


Train ANYTHING on Hugging Face Spaces hardware using AutoTrain SpaceRunner: https://hf.co/blog/stefan-it/autotrain-flair-mobie 💥

mrfakename abhishek smangrul nouamanetazi dvilasuero thomwolf clem sbrandeis


Here is my selection of papers for today (5 Jan)

https://huggingface.co/papers

Learning the 3D Fauna of the Web

Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation

Improving Diffusion-Based Image Synthesis with Context Prediction

LLaMA Pro: Progressive LLaMA with Block Expansion

LLaVA-φ: Efficient Multi-Modal Assistant with Small Language Model

Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers

FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understanding

TinyLlama: An Open-Source Small Language Model

What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs

LLM Augmented LLMs: Expanding Capabilities through Composition

ODIN: A Single Model for 2D and 3D Perception

Instruct-Imagen: Image Generation with Multi-modal Instruction

ICE-GRT: Instruction Context Enhancement by Generative Reinforcement based Transformers

Understanding LLMs: A Comprehensive Overview from Training to Inference


DreamTalk: When Expressive Talking Head Generation Meets Diffusion
  Probabilistic Models (2312.09767)

I just published a Gradio demo for AliBaba's DreamTalk 🤗

Try it now: https://huggingface.co/spaces/fffiloni/dreamtalk
Paper: https://huggingface.co/papers/2312.09767
—
DreamTalk is a diffusion-based audio-driven expressive talking head generation framework that can produce high-quality talking head videos across diverse speaking styles. DreamTalk exhibits robust performance with a diverse array of inputs, including songs, speech in multiple languages, noisy audio, and out-of-domain portraits.

cnmoro gz888 reach-vb osanseviero abidlabs radames nouamanetazi victor clem julien-c merve lunarflu IbrahimSaidi sbrandeis

santiviquez osanseviero alvarobartt reach-vb clem julien-c merve lunarflu akhaliq pierrci sbrandeis


Here is my selection of papers for today (4 Jan)

https://huggingface.co/papers

Efficient Hybrid Zoom using Camera Fusion on Mobile Phones

Incremental FastPitch: Chunk-based High Quality Text to Speech

CoMoSVC: Consistency Model-based Singing Voice Conversion

SIGNeRF: Scene Integrated Generation for Neural Radiance Fields

From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations

aMUSEd: An Open MUSE Reproduction

Image Sculpting: Precise Object Editing with 3D Geometry Control

A Vision Check-up for Language Models

Multilingual Instruction Tuning With Just a Pinch of Multilinguality

WordArt Designer API: User-Driven Artistic Typography Synthesis with Large Language Models on ModelScope

Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditions

GPT-4V(ision) is a Generalist Web Agent, if Grounded

mwz davanstrien nouamanetazi mlninad m-ric victor merve


Currently attempting to hack EvoDiff to generate binders for target proteins with some interesting results. The generated binders tend to change conformation, sometimes drastically, when bound to the target proteins compared to their unbound states. Below is the target protein with an IDR linker, the generated binder, and the binder bound to the target protein with the IDR linker structure as predicted by ESMFold. Notice how the binder goes from being a solid alpha-helix, to being beta-sheets (in orange). That's quite a change in tertiary structure!

osanseviero reach-vb victor nouamanetazi clem mlninad Jemseen mishig merve radames thomwolf samusenps vicgalle ScienceStanley sbrandeis

clefourrier alielfilali01 thomwolf samusenps ScienceStanley assiduous006


Here is my selection of papers for today (3 Jan) on Hugging Face

paper pages: https://huggingface.co/papers

En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data

Boundary Attention: Learning to Find Faint Boundaries at Any Resolution

Taming Mode Collapse in Score Distillation for Text-to-3D Generation

LLaMA Beyond English: An Empirical Study on Language Capability Transfer

Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models

A Comprehensive Study of Knowledge Editing for Large Language Models

Q-Refine: A Perceptual Quality Refiner for AI-Generated Image

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

DocLLM: A layout-aware generative language model for multimodal document understanding

VideoDrafter: Content-Consistent Multi-Scene Video Generation with LLM

TrailBlazer: Trajectory Control for Diffusion-Based Video Generation

Tonic Chunte osanseviero nouamanetazi mlninad merve diwank


Microsoft: Improving Text Embeddings with Large Language Models

- uses an LLM instead of complex pipelines to create the  training data
- directly generates data for numerous text embedding tasks
- fine tunes standard models with contrastative loss achieving great performance
- critical thought: isn't this kinda benchmark hacking? If the benchmarks are so encompassing that they capture the complete idea of embedding, it's maybe a good idea, but often it is oversimplifying, I find.

Feel free to share your thoughts, even if they like mine don't beat the benchmarks ;P

https://arxiv.org/abs/2401.00368

osanseviero pcuenq julien-c alvarobartt nouamanetazi mlninad dvilasuero Praise2112 merve sbrandeis

osanseviero julien-c davanstrien dvilasuero


Here is my selection of papers for today (2 Jan) on Hugging Face

https://huggingface.co/papers

SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity

Boosting Large Language Model for Speech Synthesis: An Empirical Study

COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training

Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws

GeoGalactica: A Scientific Large Language Model in Geoscience

Improving Text Embeddings with Large Language Models

Unicron: Economizing Self-Healing LLM Training at Scale

nouamanetazi merve clem samusenps AdinaY Tonic


✅ Ever wondered how to measure transparency in model development?

My last open-source contribution for 2023 is s Space that allows you to self-assess the transparency of your model based on the 100 indicators of the Foundation Model Transparency Index (FMTI).

The original study evaluated the developers of 10 top LLMs. Curious about how yours measures up? 👀

https://huggingface.co/spaces/mariagrandury/fmti-transparency-self-assessment

Let's commit to a 2024 with greater transparency in the AI ecosystem! 🚀

lbourdois victor osanseviero abhishek Wauplin julien-c davanstrien nouamanetazi dvilasuero merve MaziyarPanahi clem afrideva sbrandeis

osanseviero victor julien-c dvilasuero merve clem


Here is my selection of papers for today

https://huggingface.co/papers

Compact Neural Graphics Primitives with Learned Hash Probing

Restoration by Generation with Constrained Priors
SSR-Encoder: Encoding Selective Subject

Representation for Subject-Driven Generation

Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object Structure via HyperNetworks

InsActor: Instruction-driven Physics-based Characters
Unsupervised Universal Image Segmentation

Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis

DreamGaussian4D: Generative 4D Gaussian Splatting

City-on-Web: Real-time Neural Rendering of Large-scale Scenes on the Web

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision

DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaption by Combining 3D GANs and Diffusion Priors

Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action

Prompt Expansion for Adaptive Text-to-Image Generation

PanGu-Draw

I2V-Adapter: A General Image-to-Video Adapter for Video Diffusion Models

The LLM Surgeon

MathPile: A Billion-Token-Scale Pretraining Corpus for Math

MobileVLM : A Fast, Reproducible and Strong Vision Language Assistant for Mobile Devices

TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones

osanseviero Chunte victor teowu nouamanetazi aaditya sbrandeis


💬 Notux 8x7b has already its own Chat UI running on 🤗 Spaces! Feel free to give it a try and chat with Notux, and let us know how it goes.

https://huggingface.co/spaces/argilla/notux-chat-ui

Kudos to @gabrielmbmb!

gabrielmbmb osanseviero mishig Chunte ceyda sayakpaul thomwolf nouamanetazi dvilasuero pcuenq NovoCode

victor osanseviero Chunte dvilasuero pcuenq NovoCode sbrandeis


Repurposing Diffusion-Based Image Generators for Monocular Depth
  Estimation (2312.02145)

Introducing Marigold 🌼 - a universal monocular depth estimator, delivering incredibly sharp predictions in the wild! Based on Stable Diffusion, it is trained with synthetic depth data only and excels in zero-shot adaptation to real-world imagery. Check it out:

🤗 Hugging Face Space: https://huggingface.co/spaces/toshas/marigold
🤗 Hugging Face Model: https://huggingface.co/Bingxin/Marigold
🤗 Hugging Face Paper: https://huggingface.co/papers/2312.02145
🌐 Website: https://marigoldmonodepth.github.io
👾 Code: https://github.com/prs-eth/marigold
👾 Code: `pip install diffusers` (check comments to this post for details!)
📄 Paper: https://arxiv.org/abs/2312.02145

Brought to you by the fantastic team from the Photogrammetry and Remote Sensing group of ETH Zurich: Bingxin Ke (@Bingxin), Anton Obukhov (@toshas), Shengyu Huang, Nando Metzger (@nandometzger), Rodrigo Caye Daudt, and Konrad Schindler.

osanseviero akhaliq nandometzger victor toshas pwais Chunte radames sayakpaul Wauplin Tonic nouamanetazi julien-c AdinaY clem Citaman abidlabs samusenps ixaxaar sbrandeis

osanseviero akhaliq nandometzger toshas Chunte radames Wauplin julien-c clem sbrandeis

osanseviero akhaliq toshas Chunte julien-c clem Tonic sbrandeis


Do you use Python to visualize data? Wouldn't it be nice if an AI chatbot can help you write Python code and improve your visualization automatically?

Check out our new blog post on how to build an AI chatbot to run code and tweak plots: https://huggingface.co/blog/sophiamyang/tweak-mpl-chat

sophiamyang philippjfr osanseviero victor kramp ahuang11 Chunte MarcSkovMadsen droumis pcuenq nouamanetazi julien-c merve sbrandeis YAYA20

Chunte MarcSkovMadsen droumis dleybel julien-c samusenps sourceoftruthdata sbrandeis YAYA20

Chunte MarcSkovMadsen droumis julien-c sbrandeis YAYA20


kramp Citaman ariG23498 mwitiderrick abhishek akashicmarga rishiraj mergisi pwais Chunte gpt3eth VictorSanh nouamanetazi julien-c lunarflu BrigitteTousi radames JackCloudman yuchenlin clem antiven0m santiviquez


some mandatory hello world as first post 🤗

```
from tokenizers import Tokenizer
tokenizer = Tokenizer.from_pretrained("bert-base-uncased")
tokenizer.encode("Hello world~ Hello 2024").tokens
```

chansung PSegs osanseviero kramp abhishek mergisi victor Chunte radames Tonic nouamanetazi mishig julien-c samusenps benvdv rreed Sengil

mariagrandury Chunte radames pcuenq mishig julien-c clefourrier gabrielchua rreed


Here is my selection of papers for today (27 Dec) on Hugging Face daily papers newsletter

daily pagers feed: https://huggingface.co/papers

UniRef++: Segment Every Reference Object in Spatial and Temporal Spaceshttps://huggingface.co/papers/2312.15715

LangSplat: 3D Language Gaussian Splattinghttps://huggingface.co/papers/2312.16084

Human101: Training 100+FPS Human Gaussians in 100s from 1 Viewhttps://huggingface.co/papers/2312.15258

Audiobox: Unified Audio Generation with Natural Language Promptshttps://huggingface.co/papers/2312.15821

HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3Dhttps://huggingface.co/papers/2312.15980

One-dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationshttps://huggingface.co/papers/2312.16145

Make-A-Character: High Quality Text-to-3D Character Generation within Minuteshttps://huggingface.co/papers/2312.15430

A Recipe for Scaling up Text-to-Video Generation with Text-free Videoshttps://huggingface.co/papers/2312.15770

Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4https://huggingface.co/papers/2312.16171

SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalinghttps://huggingface.co/papers/2312.15166

Supervised Knowledge Makes Large Language Models Better In-context Learnershttps://huggingface.co/papers/2312.15918

Gemini vs GPT-4V: A Preliminary Comparison and Combination of Vision-Language Models Through Qualitative Caseshttps://huggingface.co/papers/2312.15011

clefourrier osanseviero gz888 Decombas 0x90e ceyda wchai ClayFace mariagrandury Chunte victor nouamanetazi dvilasuero sbrandeis

victor osanseviero radames gz888 Decombas wchai rbiswasfc mwitiderrick abhishek Chunte Tonic dvilasuero


💨 Notux 8x7b was just released!

From Argilla, we recently fine-tuned Mixtral 8x7b Instruct from Mistral AI using DPO, and a binarized and curated version of UltraFeedback, to find out it outperforms every other MoE-based model on the Hub.

- https://huggingface.co/argilla/notux-8x7b-v1
- https://huggingface.co/datasets/argilla/ultrafeedback-binarized-preferences-cleaned

osanseviero gabrielmbmb plaguss 1littlecoder dvilasuero gsarti julien-c davidefiocco eliolio davanstrien wgpubs pierrci nouamanetazi abhishek raulmarindev iarbel AdjectiveAllison dariolopez datablets Chunte svilupp doolayer jackvial mishig cbensimon jeffboudier victor merve d0rj samusenps Yudiss sbrandeis

gabrielmbmb plaguss victor osanseviero radames dvilasuero julien-c davanstrien DamarJati abhishek raulmarindev Chunte mishig the-qa-company sbrandeis

dvilasuero julien-c davanstrien osanseviero raulmarindev Chunte mishig sbrandeis

kramp mwitiderrick raulmarindev Chunte MoritzLaurer mishig sbrandeis


lbourdois osanseviero pierrci santiviquez atulyaatul glenn-parham neovalle PetraLB gsarti mrfakename himanshubeniwal tunguz Felladrin mariagrandury jefferylovely Wauplin sandrinjoy thibaultM clefourrier radames sophiamyang alielfilali01 davanstrien abhishek ybelkada josers18 joeddav thomwolf DGoradia ceyda qqliangqi chansung rishiraj mwz shivance victor clem susnato lazarustda Javiai mcpotato Chunte themanas021 adarshxs Kirti jarvisx17 nouamanetazi lysandre TuringsSolutions Midgardsormr Tanvir1337 mishig stubbi sbrandeis

PetraLB osanseviero himanshubeniwal tunguz mariagrandury Wauplin radames sophiamyang DGoradia pain KnutJaegersberg beratcmn clem susnato Chunte themanas021 patrickfleith Midgardsormr Tanvir1337 stubbi

osanseviero sophiamyang pain porar18837 clem Chunte themanas021 himanshubeniwal

himanshubeniwal tunguz sophiamyang ybelkada clem Chunte themanas021


just setting up my new hf social posts account feature 🤗

jbilcke-hf linoyts osanseviero davanstrien pierrci Wauplin clefourrier radames gz888 KnutJaegersberg Chunte Outer-Spatial nouamanetazi clem VictorSanh abidlabs

mergisi Chunte Tonic clem rwightman abidlabs Epiculous


Holiday talk about AI taking over? Let's shift the narrative!

🌟 There is no reason to believe that just because AI systems are intelligent they will want to dominate us. Yann LeCun reminds us that AI systems won't have the same motivations as humans, we'll design them not to.

🌍 Instead of getting distracted by future existential risks, we must address AI’s more pressing risks — like emitting carbon, infringing copyrights and spreading bias. Sasha Luccioni urges us to create tools and legislation that promote transparency and diversity.

💡 Dive deeper into these perspectives:
- Yann's (@ylecun) WIRED interview (12'): https://www.wired.com/story/artificial-intelligence-meta-yann-lecun-interview/
- Sasha's (@sasha) TED Talk (10'): https://www.ted.com/talks/sasha_luccioni_ai_is_dangerous_but_not_for_the_reasons_you_think

P.S.: Love this new "Posts" feature, big thanks to 🤗 for letting me try it!

What are your go-to citations for AI risks? 👇

lbourdois victor osanseviero davanstrien finiteautomata jbilcke-hf pierrci clefourrier mariagrandury radames julien-c ceyda BramVanroy Chunte nouamanetazi samusenps interstellarninja dvilasuero sbrandeis

victor osanseviero kramp julien-c Chunte neovalle mvaloatto dvilasuero sbrandeis

julien-c Chunte samusenps dvilasuero sbrandeis


Cool feature! Thanks, HF, for allowing me test it.

kramp osanseviero mariagrandury davanstrien SaylorTwift victor lbourdois pierrci julien-c ceyda Chunte radames nouamanetazi multimodalart pfung SkalskiP VictorSanh mvaloatto d0rj sbrandeis GPT007

victor osanseviero julien-c Chunte radames Tonic multimodalart ybelkada

osanseviero Wauplin Chunte afrideva multimodalart Paulgramma


By far the coolest release of the day!
> The Open LLM Leaderboard, most comprehensive suite for comparing Open LLMs on many benchmarks, just released a comparator tool that lets you dig into the detail of differences between any models.

Here's me checking how the new Llama-3.1-Nemotron-70B that we've heard so much compares to the original Llama-3.1-70B. 🤔🔎

Try it out here 👉 https://huggingface.co/spaces/open-llm-leaderboard/comparator


🚨 We’ve just released a new tool to compare the performance of models in the 🤗 Open LLM Leaderboard: the Comparator 🎉
https://huggingface.co/spaces/open-llm-leaderboard/comparator

Want to see how two different versions of LLaMA stack up? Let’s walk through a step-by-step comparison of LLaMA-3.1 and LLaMA-3.2. 🦙🧵👇

1/ Load the Models' Results
- Go to the 🤗 Open LLM Leaderboard Comparator: https://huggingface.co/spaces/open-llm-leaderboard/comparator
- Search for "LLaMA-3.1" and "LLaMA-3.2" in the model dropdowns.
- Press the Load button. Ready to dive into the results!

2/ Compare Metric Results in the Results Tab 📊
- Head over to the Results tab.
- Here, you’ll see the performance metrics for each model, beautifully color-coded using a gradient to highlight performance differences: greener is better! 🌟
- Want to focus on a specific task? Use the Task filter to hone in on comparisons for tasks like BBH or MMLU-Pro.

3/ Check Config Alignment in the Configs Tab ⚙
- To ensure you’re comparing apples to apples, head to the Configs tab.
- Review both models’ evaluation configurations, such as metrics, datasets, prompts, few-shot configs...
- If something looks off, it’s good to know before drawing conclusions! ✅

4/ Compare Predictions by Sample in the Details Tab 🔍
- Curious about how each model responds to specific inputs? The Details tab is your go-to!
- Select a Task (e.g., MuSR) and then a Subtask (e.g., Murder Mystery) and then press the Load Details button.
- Check out the side-by-side predictions and dive into the nuances of each model’s outputs.

5/ With this tool, it’s never been easier to explore how small changes between model versions affect performance on a wide range of tasks. Whether you’re a researcher or enthusiast, you can instantly visualize improvements and dive into detailed comparisons.

🚀 Try the 🤗 Open LLM Leaderboard Comparator now and take your model evaluations to the next level!


This has to be the first peak performance level use case of a non-autoregressive architecture for TTS. Flow matching for the win!!

Demo: https://huggingface.co/spaces/mrfakename/E2-F5-TTS
Model: https://huggingface.co/SWivid/E2-TTS


All the way from Korea, a novel approach called Mentor-KD significantly improves the reasoning abilities of small language models.

Mentor-KD introduces an intermediate-sized "mentor" model to augment training data and provide soft labels during knowledge distillation from large language models (LLMs) to smaller models.

Broadly, it’s a two-stage process:
1) Fine-tune the mentor on filtered Chain-of-Thought (CoT) annotations from an LLM teacher.
2) Use the mentor to generate additional CoT rationales and soft probability distributions.

The student model is then trained using:
- CoT rationales from both the teacher and mentor (rationale distillation).
- Soft labels from the mentor (soft label distillation).

Results show that Mentor-KD consistently outperforms baselines, with up to 5% accuracy gains on some tasks.

Mentor-KD is especially effective in low-resource scenarios, achieving comparable performance to baselines while using only 40% of the original training data.

This work opens up exciting possibilities for making smaller, more efficient language models better at complex reasoning tasks.

What are your thoughts on this approach?


https://huggingface.co/nvidia/Llama-3.1-Nemotron-70B-Instruct-HF is now available in HuggingChat if you want to try it!

https://huggingface.co/chat/models/nvidia/Llama-3.1-Nemotron-70B-Instruct-HF

KingNish prithivMLmods jgitsolutions John6666 adorkin AtAndDev clem


🚨 I have $3,500 in Azure credits, including access to an H100 (96 Go), expiring on November 12, 2024.

I won’t be able to use it all myself, so I’m reaching out to the @huggingface community: Are there any open-source projets with data ready for some compute power?

 Let’s collaborate and make the most of it together 🔗


I am here to provide you with the premium codes you want just by informing me your requirements. #program #AI #code
Premium_Code
https://hf.co/chat/assistant/670fd7b2ad7fdbb38ff98102


Hi there HuggingFacers!

Have you ever dreamt of an improbable books crossover, like Frodo from 𝘓𝘰𝘳𝘥 𝘰𝘧 𝘵𝘩𝘦 𝘙𝘪𝘯𝘨𝘴 becoming the main character of the 𝘖𝘥𝘺𝘴𝘴𝘦𝘺 or Emma Bovary from 𝘔𝘢𝘥𝘢𝘮𝘦 𝘉𝘰𝘷𝘢𝘳𝘺 acting as a modern-days Shakespearean Juliet?

Well, all of this is now possible! I'm thrilled to introduce my latest opensource product for storytelling: 𝐛𝐨𝐨𝐤𝐬-𝐦𝐢𝐱𝐞𝐫-𝐚𝐢 𝐯𝟎.𝟎.𝟎 !

Built with ReactJS and shipped directly to you on Spaces thanks to Docker, this webapp combines the power of two AI tools:

- gpt-4o-mini by OpenAI, which takes care of cooking new and intriguing plots starting from the user's instructions, the titles and the summaries of the two books to mix (summaries are scraped through Wikipedia)
- text2img realtime API by ModelsLab, which provides a stable diffusion pipeline to create a thumbnail for your newly-generated story

Everything is provided under a simple and intuitive UI, which uses chatscope's React template kit.
Curious of trying? The app is already live at:

https://huggingface.co/spaces/as-cle-bert/books-mixer-ai

And you can also have a tour of the GitHub repo (and leave a little ⭐ while you're there):

https://github.com/AstraBert/books-mixer-ai

The documentation is still under construction, but will become available soon😊

Have fun!📚📚


Today I was able to solve a very difficult coding session with GPT-4o which ended up solving integrations on a very large scale.   So I decided to look a bit more into how its reasoners work.  Below is a fun markdown emoji outline about what I learned today and what I'm pursuing.

Hope you enjoy!  Cheers, Aaron.

Also here are my favorite last 4 spaces I am working on:
1. GPT4O:  https://huggingface.co/spaces/awacke1/GPT-4o-omni-text-audio-image-video
2. Claude:
 https://huggingface.co/spaces/awacke1/AnthropicClaude3.5Sonnet-ACW
3. MSGraph M365: https://huggingface.co/spaces/awacke1/MSGraphAPI
4. Azure Cosmos DB: Now with Research AI!  https://huggingface.co/spaces/awacke1/AzureCosmosDBUI

# 🚀 OpenAI's O1 Models: A Quantum Leap in AI

## 1. 🤔 From 🦜 to 🧠: O1's Evolution

- **Thinking AI**: O1 ponders before replying; GPT models just predict. 💡

## 2. 📚 AI Memory: 💾 + 🧩 = 🧠

- **Embeddings & Tokens**: Words ➡ vectors, building knowledge. 📖

## 3. 🔍 Swift Knowledge Retrieval

- **Vector Search & Indexing**: O1 finds info fast, citing reliable sources. 🔎📖

## 4. 🌳 Logic Trees with Mermaid Models

- **Flowchart Reasoning**: O1 structures thoughts like diagrams. 🎨🌐

## 5. 💻 Coding Mastery

- **Multilingual & Current**: Speaks many code languages, always up-to-date. 💻🔄

## 6. 🏆 Breaking Records

- **92.3% MMLU Score**: O1 outperforms humans, setting new AI standards. 🏅

## 7. 💡 Versatile Applications

- **Ultimate Assistant**: From fixing code to advancing research. 🛠🔬

## 8. 🏁 Racing Toward AGI

- **OpenAI Leads**: O1 brings us closer to true AI intelligence. 🚀

## 9. 🤖 O1's Reasoning Pillars

- **🧠 Chain of Thought**: Step-by-step logic.
- **🎲 MCTS**: Simulates options, picks best path.
- **🔍 Reflection**: Self-improves autonomously.
- **🏋♂ Reinforcement Learning**: Gets smarter over time.

---

*Stay curious, keep coding!* 🚀


In regards to the latest mistral model and GGUFs for it:

Yes, they may be subpar and may require changes to llama.cpp to support the interleaved sliding window

Yes, I got excited when a conversion worked and released them ASAP

That said, generation seems to work right now and seems to mimic the output from spaces that are running the original model

I have appended -TEST to the model names in an attempt to indicate that they are not final or perfect, but if people still feel mislead and that it's not the right thing to do, please post (civilly) below your thoughts, I will highly consider pulling the conversions if that's what people think is best. After all, that's what I'm here for, in service to you all !


🔥 Meta AI just blessed us with CoTracker v3, bleeding edge point tracking foundation model 🤩
model: https://huggingface.co/facebook/cotracker3
demo: https://huggingface.co/spaces/facebook/cotracker

John6666 jgitsolutions KvrParaskevi Rostenbach LeonceNsh


Today I found out about the existence of https://huggingface.co/utter-project/EuroLLM-1.7B-Instruct and unexpectedly it is really good. I think it's a very underrated model - give it a try https://huggingface.co/spaces/nyuuzyou/EuroLLM-1.7B-Instruct


Ever wondered how neural networks actually work under the hood?

In my latest video, I break down the core mathematical concepts behind neural networks in a way that's easy for IT professionals to understand. We'll explore:

- Neurons as logic gates
- Weighted sums and activation functions
- Gradient descent and backpropagation

No complex equations or jargon, just clear explanations and helpful visuals!

➡ Watch now and unlock the mysteries of neural networks: https://youtu.be/L5_I1ZHoGnM


Philosopher Gilles Deleuze in 1985-86 about society of control, probabilities, and power. Visionary words in an era of autoregressive models:

"The biopolitics of populations appears when right sets about administering life, says Foucault, administering life in any open multiplicities whatever. You see the importance of the difference between discipline and biopolitics. The one is in an open space, with large multiplicities to which limits are not assignable. They can only be treated by the calculus of probabilities, hence the development of the calculus of probabilities and the meaning [sens] of the social control of probabilities, the probabilities of marriage in a nation, the probabilities of mortality, probabilities of natality. Natality, nuptiality, mortality …

... When Foucault directly addresses the question of power, namely, one of his great theses: no, power does not repress, or it represses only secondarily. What does it do? It does something much more profound and, doubtless, more formidable that repressing: it forms, it shapes. It does not silence, it does worse: it makes speak. It disciplines, it standardizes [normalise]. But repression is entirely secondary in relation to the positive operations of power.

Power does not repress, it disciplines, it manages, it controls, it standardizes, etcetera. It does not silence, it makes speak. It does not prevent acting, it makes act."

From the Deleuze Seminars at Université Paris 8 translated by Purdue University -> https://deleuze.cla.purdue.edu/


The Synthetic Data Generator now directly integrates with Argilla, so you can generate and curate your own high-quality datasets from pure natural language!

Up next -> include dataset generation for text classification.
Other suggestions? Let us know.

Space: https://huggingface.co/spaces/argilla/synthetic-data-generator


While Google's Transformer might have introduced "Attention is all you need," Microsoft and Tsinghua University are here with the DIFF Transformer, stating, "Sparse-Attention is all you need."

The DIFF Transformer outperforms traditional Transformers in scaling properties, requiring only about 65% of the model size or training tokens to achieve comparable performance.

The secret sauce? A differential attention mechanism that amplifies focus on relevant context while canceling out noise, leading to sparser and more effective attention patterns.

How?
- It uses two separate softmax attention maps and subtracts them.
- It employs a learnable scalar λ for balancing the attention maps.
- It implements GroupNorm for each attention head independently.
- It is compatible with FlashAttention for efficient computation.

What do you get?
- Superior long-context modeling (up to 64K tokens).
- Enhanced key information retrieval.
- Reduced hallucination in question-answering and summarization tasks.
- More robust in-context learning, less affected by prompt order.
- Mitigation of activation outliers, opening doors for efficient quantization.

Extensive experiments show DIFF Transformer's advantages across various tasks and model sizes, from 830M to 13.1B parameters.

This innovative architecture could be a game-changer for the next generation of LLMs. What are your thoughts on DIFF Transformer's potential impact?


MixGen3 is an innovative image generation service that utilizes LoRA (Low-Rank Adaptation) models. Its key features include:

Integration of various LoRA models: Users can explore and select multiple LoRA models through a gallery.
Combination of LoRA models: Up to three LoRA models can be combined to express unique styles and content.
User-friendly interface: An intuitive interface allows for easy model selection, prompt input, and image generation.
Advanced settings: Various options are provided, including image size adjustment, random seed, and advanced configurations.

Main applications of MixGen3:

Content creation
Design and illustration
Marketing and advertising
Education and learning

Value of MixGen3:

Enhancing creativity
Time-saving
Collaboration possibilities
Continuous development

Expected effects:

Increased content diversity
Lowered entry barrier for creation
Improved creativity
Enhanced productivity

MixGen3 is bringing a new wave to the field of image generation by leveraging the advantages of LoRA models. Users can experience the service for free at
https://openfree-mixgen3.hf.space

contacts: arxivgpt@gmail.com

aiqtech seawolf2357 fantos aiqcamp ginipick openfree DeathGodlike Sakalti Moibe

aiqtech seawolf2357 fantos aiqcamp ginipick openfree Ar4ikov

aiqtech seawolf2357 fantos fantaxy openfree John6666

aiqtech seawolf2357 aiqcamp ginipick fantaxy openfree

aiqtech seawolf2357 fantos ginipick openfree

aiqtech seawolf2357 fantos aiqcamp openfree


Model is always disabled?
#script...
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("distilbert/distilgpt2",
                                             token="xxxxxx")

That loads the model fine.  But if used by index returned from VectorStoreIndex for QDrant like this:

#script...
query_engine = index_from_nodes.as_query_engine(llm=model, streaming=True)

response = query_engine.query(
    "What is formula 1?"
)

response.print_response_stream()

It errors out with a disabled error:
AssertionError                            Traceback (most recent call last)
Cell In[34], line 1
----> 1 query_engine = index_from_nodes.as_query_engine(llm=model, streaming=True)
      3 response = query_engine.query(
      4     "What is formula 1?"
      5 )
      7 response.print_response_stream()

File ~/miniconda/lib/python3.9/site-packages/llama_index/core/indices/base.py:376, in BaseIndex.as_query_engine(self, llm, **kwargs)
    370 from llama_index.core.query_engine.retriever_query_engine import (
    371     RetrieverQueryEngine,
    372 )
    374 retriever = self.as_retriever(**kwargs)
    375 llm = (
--> 376     resolve_llm(llm, callback_manager=self._callback_manager)
    377     if llm
    378     else Settings.llm
    379 )
    381 return RetrieverQueryEngine.from_args(
    382     retriever,
    383     llm=llm,
    384     **kwargs,
    385 )

File ~/miniconda/lib/python3.9/site-packages/llama_index/core/llms/utils.py:102, in resolve_llm(llm, callback_manager)
     99     print("LLM is explicitly disabled. Using MockLLM.")
    100     llm = MockLLM()
--> 102 assert isinstance(llm, LLM)
    104 llm.callback_manager = callback_manager or Settings.callback_manager
    106 return llm

AssertionError:

So why is the LLM disabled?
Thanks!


📢 We are giving extra two weeks before switching to the final stage of RuOpinionNE-2024.
⏰  The final stage starts since 1-st of November 2024.
We have already first baseline submission by 👨💻 RefalMachine that showcase F1 = 0.17  based on Qwen2 model series.

For those who wish to attend:
📊 Codalab: https://codalab.lisn.upsaclay.fr/competitions/20244
🗒 Task: https://codalab.lisn.upsaclay.fr/competitions/20244#learn_the_details-overview
🔔 Updates: https://t.me/RuOpinionNE2024

🙋 Questions: https://nicolay-r.github.io/
🧪 Past experiments: https://github.com/nicolay-r/RuSentNE-LLM-Benchmark


Professors should ask students to write blog posts based on their final projects instead of having them do paper-like reports.

A single blog post, accessible to the entire internet, can have a greater career impact than dozens of reports that nobody will read.


Who's going to get to the most liked model on Hugging Face first: StabilityAI, Meta, Black Forest or someone else? The race is on!


New Book: Building Disruptive AI & LLM Technology from Scratch https://mltblog.com/404F1BZ

This book features new advances in game-changing AI and LLM technologies built by GenAItechLab.com. Written in simple English, it is best suited for engineers, developers, data scientists, analysts, consultants and anyone with an analytic background interested in starting a career in AI. The emphasis is on scalable enterprise solutions, easy to implement, yet outperforming vendors both in term of speed and quality, by several orders of magnitude.

Each topic comes with GitHub links, full Python code, datasets, illustrations, and real-life case studies, including from Fortune 100 company. Some of the material is presented as enterprise projects with solution, to help you build robust applications and boost your career. You don’t need expensive GPU and cloud bandwidth to implement them: a standard laptop works.

➡ Part 1: Hallucination-Free LLM with Real-Time Fine-Tuning

➡ Part 2: Outperforming Neural Nets and Classic AI

➡ Part 3: Innovations in Statistical AI

About the author

Vincent Granville is a pioneering GenAI scientist and machine learning expert, co-founder of Data Science Central (acquired by a publicly traded company in 2020), Chief AI Scientist at ML Techniques and GenAI Techlab, former VC-funded executive, author (Elsevier) and patent owner — one related to LLM. Vincent’s past corporate experience includes Visa, Wells Fargo, eBay, NBC, Microsoft, and CNET.

➡ See content and get your copy, at https://mltblog.com/404F1BZ


Don't use an LLM when you can use a much cheaper model.

The problem is that no one tells you how to actually do it.

Just picking a pre-trained model (e.g., BERT) and throwing it at your problem won't work!

If you want a small model to perform well on your problem, you need to fine-tune it.

And to fine-tune it, you need data.

The good news is that you don't need a lot of data but instead high-quality data for your specific problem.

In the latest livestream, I showed you guys how to get started with Argilla on the Hub! Hope to see you at the next one.

https://www.youtube.com/watch?v=BEe7shiG3rY


AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents (2410.09024)

🚨 New Agent Benchmark 🚨
AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents

https://huggingface.co/datasets/ai-safety-institute/AgentHarm

Collaboration between UK AI Safety Institute and Gray Swan AI to create a dataset for measuring harmfulness of LLM agents.

The benchmark contains both harmful and benign sets of 11 categories with varied difficulty levels and detailed evaluation, not only testing success rate but also tool level accuracy.

We provide refusal and accuracy metrics across a wide range of models in both no attack and prompt attack scenarios.

https://huggingface.co/papers/2410.09024


New York Times to Perplexity: Stop Using Our Stuff

The publisher has sent generative-AI startup Perplexity a “cease and desist” notice demanding that the firm stop accessing and using its content, according to a copy of the letter reviewed by The Wall Street Journal.

Perplexity CEO Aravind Srinivas said in an interview that Perplexity isn’t ignoring the Times’s efforts to block crawling of its site. He said the company plans on responding to the legal notice by the Times’s deadline of Oct. 30.

“We are very much interested in working with every single publisher, including the New York Times,” Srinivas said. “We have no interest in being anyone’s antagonist here.”

https://www.wsj.com/business/media/new-york-times-to-bezos-backed-ai-startup-stop-using-our-stuff-20faf2eb?mod=rss_Technology


I just published the first article in a pair. I could make it a longer tailed series, in case you liked em. This one dives into self-hosting Metaflow without needing S3, illustrated with a version tailored for Google Colab.
find it @ https://huggingface.co/blog/Aurelien-Morgan/stateful-metaflow-on-colab


👋 We're excited to share our project on mobile face detection using MediaPipe and ZETIC.MLange.

🔑 Key highlights:
1. Introduction to Mediapipe face detection model
2. Developing on-device AI applications with ZETIC.MLange
3. Guide to creating object detection apps utilizing Mobile NPUs

📱 Learn how to build a high-performance face detection app that operates entirely on-device, no cloud required! We explore:
- Real-time face analysis techniques
- Enhanced security measures
- Privacy protection strategies

🎥 Check out our demo video showcasing real-time face detection:
Face detection Demo: https://youtu.be/GXtJKk7MdjQ?si=41UFKL8IBwx5nlxs

📚 Full Tutorial
For a step-by-step guide and in-depth discussion, read our full blog post:
https://zetic.ai/blog/implementing-face-detection-on-device-ai-with-zetic-mlange

We'd love to hear your thoughts, experiences, or questions in the comments below.


Good folks from Universitat Politècnica de Catalunya, University of Groningen, and Meta have released "A Primer on the Inner Workings of Transformer-based Language Models."

They don't make survey papers like they used to, but this is an exciting new survey on Transformer LM interpretability!

This comprehensive survey provides a technical deep dive into:

• Transformer architecture components (attention, FFN, residual stream)
• Methods for localizing model behavior:
  - Input attribution (gradient & perturbation-based)
  - Component importance (logit attribution, causal interventions)
• Information decoding techniques:
  - Probing, linear feature analysis
  - Sparse autoencoders for disentangling features
• Key insights on model internals:
  - Attention mechanisms (induction heads, copy suppression)
  - FFN neuron behaviors
  - Residual stream properties
  - Multi-component emergent behaviors

The paper offers a unified notation and connects insights across different areas of interpretability research. It's a must-read for anyone working on understanding large language models!

Some fascinating technical highlights:
- Detailed breakdowns of attention head circuits (e.g., IOI task)
- Analysis of factual recall mechanisms
- Overview of polysemanticity and superposition
- Discussion of grokking as circuit emergence

What interpretability insights do you find most intriguing?


Transformers are not all we need, that is being proven repeatedly now as more alternative frameworks emerge. Another such framework is Kolmogorov Arnold Network based Transformers. I break down exactly how these differ from Perceptron based Transformers and give you the link to my Colab where I create a model based on the research paper that absolutely destroys a standard Transformers based model. Check out the video here: https://www.youtube.com/watch?v=Sw0euxNZCc4


To demonstrate that it was possible, I performed a "trapezoid" gradient merge of a Llama 3 8B model onto Llama 3.1 8B Instruct, favoring the L3.1 model at the ends in order to preserve coherence and limiting the influence of the L3 model to at most 0.1 weight. Tested to 16k context length.
https://huggingface.co/grimjim/Llama-Nephilim-Metamorphosis-v2-8B


🎙 Introducing LiveATC Recordings (Partial 2024-08-26) Dataset - https://huggingface.co/datasets/nyuuzyou/liveatc

Dataset highlights:

- 21,172 air traffic control audio recordings from LiveATC.net for August 26, 2024
- Multilingual content, primarily in English with potential for other languages
- Each entry includes: audio file, ICAO airport code, facility type, date, and time
- Contains original MP3 files stored in .tar.zst archives, organized by ICAO airport code
- Data covers various airports and ATC facilities worldwide
- Subject to LiveATC.net's Terms of Use for personal, non-commercial use only

The dataset can be used for audio classification, automatic speech recognition, and analysis of air traffic control communications. The inclusion of recordings from multiple airports allows for comparative analysis across different locations and facility types.


𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

⚡ 𝐓𝐡𝐢𝐬 𝐦𝐨𝐧𝐭𝐡'𝐬 𝐦𝐨𝐬𝐭 𝐢𝐦𝐩𝐨𝐫𝐭𝐚𝐧𝐭 𝐛𝐫𝐞𝐚𝐤𝐭𝐡𝐫𝐨𝐮𝐠𝐡: 𝐃𝐢𝐟𝐟𝐞𝐫𝐞𝐧𝐭𝐢𝐚𝐥 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫 𝐯𝐚𝐬𝐭𝐥𝐲 𝐢𝐦𝐩𝐫𝐨𝐯𝐞𝐬 𝐚𝐭𝐭𝐞𝐧𝐭𝐢𝐨𝐧 ⇒ 𝐛𝐞𝐭𝐭𝐞𝐫 𝐫𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐚𝐧𝐝 𝐟𝐞𝐰𝐞𝐫 𝐡𝐚𝐥𝐥𝐮𝐜𝐢𝐧𝐚𝐭𝐢𝐨𝐧𝐬!

Thought that self-attention could not be improved anymore?

Microsoft researchers have dropped a novel "differential attention" mechanism that amplifies focus on relevant context while canceling out noise. It sounds like a free lunch, but it does really seem to vastly improve LLM performance!

𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

🧠 Differential attention computes the difference between two separate softmax attention maps, canceling out noise and promoting sparse attention patterns

🔥 DIFF Transformer outperforms standard Transformers while using 35-40% fewer parameters or training tokens

📏 Scales well to long contexts up to 64K tokens, leveraging increasing context length more effectively

🔎 Dramatically improves key information retrieval, enhancing in-context learning, and possibly reducing risk of hallucinations 🤯

🔢 Reduces activation outliers, potentially enabling lower-bit quantization without performance drop!

⚙ Can be directly implemented using existing FlashAttention kernels

This new architecture could lead much more capable LLMs, with vastly improved strengths in long-context understanding and factual accuracy.

But they didn’t release weights on the Hub: let’s wait for the community to train the first open-weights DiffTransformer! 🚀

Read their paper 👉 https://huggingface.co/papers/2410.05258


Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization - 15 vs 256 images having datasets compared as well (expressions / emotions tested too) - Used Kohya GUI for training

Full files and article : https://www.patreon.com/posts/112099700

Download images in full resolution to see prompts and model names

All trainings are done with Kohya GUI, perfectly can be done locally on Windows, and all trainings were 1024x1024 pixels

Fine Tuning / DreamBooth works as low as 6 GB GPUs (0 quality degrade totally same as 48 GB config)

Best quality of LoRA requires 48 GB GPUs , 24 GB also works really good and minimum 8 GB GPU is necessary for LoRA (lots of quality degrade)

Full size grids are also shared for the followings: https://www.patreon.com/posts/112099700

Additionally, I have shared full training entire logs that you can see each checkpoint took time. I have shared best checkpoints, their step count and took time according to being either LoRA, Fine Tuning or Batch size 1 or 7 or 15 images or 256 images, so a very detailed article regarding completed.

Check the images to see all shared files in the post.

Furthermore, a very very detailed analysis having article written and all latest DreamBooth / Fine Tuning configs and LoRA configs are shared with Kohya GUI installers for both Windows, Runpod and Massed Compute.

Moreover, I have shared new 28 realism and 37 stylization testing prompts.

Current tutorials are as below:

Windows requirements CUDA, Python, cuDNN, and such : https://youtu.be/DrhUHnYfwC0

How to use SwarmUI : https://youtu.be/HKX8_F1Er_w

How to use FLUX on SwarmUI : https://youtu.be/bupRePUOA18

How to use Kohya GUI for FLUX training : https://youtu.be/nySGu12Y05k

How to use Kohya GUI for FLUX training on Cloud (RunPod and Massed Compute) : https://youtu.be/-uhL2nW7Ddw


we have a leaderboard for video LLMs, and most of the top models are open ones! https://huggingface.co/spaces/opencompass/openvlm_video_leaderboard 👑👏
we are so back 🔥


Baichuan-Omni Technical Report (2410.08565)

为什么我会在作者列表上，是huggingface人名匹配的吗？

为什么我会在作者列表上，是huggingface人名匹配的吗？


I have finally completed a working full Azure and Microsoft MS Graph API implementation which can use all the interesting MS AI features in M365 products to manage CRUD patterns for the graph features across products.

This app shows initial implementation of security, authentication, scopes, and access to Outlook, Calendar, Tasks, Onedrive and other apps for CRUD pattern as AI agent service skills to integrate with your AI workflow.

Below are initial screens showing integration:

URL: https://huggingface.co/spaces/awacke1/MSGraphAPI
Discussion: https://huggingface.co/spaces/awacke1/MSGraphAPI/discussions/5

Best of AI on @Azure and @Microsoft on @HuggingFace :
https://huggingface.co/microsoft
https://www.microsoft.com/en-us/research/
---
Aaron


Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥

> WhisperSpeech X Llama 3.1 8B
> Trained on 50K hours of speech (7 languages)
> Continually trained on 45hrs 10x A1000s
> MLS -> WhisperVQ tokens -> Llama 3.1
> Instruction tuned on 1.89M samples
> 70% speech, 20% transcription, 10% text
> Apache 2.0 licensed ⚡

Architecture:
> WhisperSpeech/ VQ for Semantic Tokens
> Llama 3.1 8B Instruct for Text backbone
> Early fusion (Chameleon)

I'm super bullish on HomeBrew/ Jan and early fusion, audio and text, multimodal models!

(P.S. Play with the demo on Hugging Face: https://huggingface.co/spaces/jan-hq/Ichigo-llama3.1-s-instruct)

vigos YaTharThShaRma999 chenduo Yadukrishnan edwixx


Just started going through the latest "State of AI Report 2024", and I cannot get over the predictions!

The report predicts major developments in AI over the next 12 months, including a $10B+ investment from a sovereign state into a large US AI lab, triggering national security scrutiny, and a viral app created by someone without coding skills.

It forecasts changes in data collection practices due to frontier labs facing trials, softer-than-expected EU AI Act implementations, and the rise of an open-source alternative to OpenAI GPT-4 outperforming in benchmarks.

NVIDIA’s dominance will remain largely unchallenged, investment in humanoid robots will decline, Apple’s on-device AI research will gain momentum, and a research paper by an AI scientist will be accepted at a major conference.

Lastly, a GenAI-based video game is expected to achieve breakout success.

Yet to go through all 200+ pages... will post summarized thoughts later.


she assert on my device until i give up AHAHEGHFDGHJHASUFSHD


MiniSearch is celebrating its 1st birthday! 🎉

Exactly one year ago, I shared the initial version of this side-project on Hugging Face. Since then, there have been numerous changes under the hood. Nowadays it uses [Web-LLM](https://github.com/mlc-ai/web-llm), [Wllama](https://github.com/ngxson/wllama) and [SearXNG](https://github.com/searxng/searxng). I use it daily as my default search engine and have done my best to make it useful. I hope it's interesting for you too!

HF Space: https://huggingface.co/spaces/Felladrin/MiniSearch
Embeddable URL: https://felladrin-minisearch.hf.space

Locutusque dillfrescott shtefcs John6666 Zmu Nymbo DeathGodlike KatyaKnw


🎓 Introducing Svitppt.com.ua Presentations Dataset - https://huggingface.co/datasets/nyuuzyou/svitppt

Dataset highlights:
- 18,001 presentations from svitppt.com.ua, a platform for storing and viewing presentations for Ukrainian school students
- Primarily in Ukrainian, with some Russian and English content
- Each entry includes: URL, title, download URL, filepath, and extracted text content (where available)
- Contains original PPT/PPTX files in addition to metadata
- Data covers a wide range of educational topics and presentation materials
- Dedicated to the public domain under Creative Commons Zero (CC0) license

The dataset can be used for analyzing educational presentation content in Ukrainian and other languages, text classification tasks, and information retrieval systems. It's particularly valuable for examining trends in educational presentation materials and sharing practices in the Ukrainian-speaking student community. The inclusion of original files allows for in-depth analysis of presentation formats and structures commonly used in Ukrainian educational settings.


The question is on the left and the answer is on the right.


Is Entropix the Chain of Thought Reasoning method behind GPTo1? Using a mixture of entropy, varentropy, and prompt engineering, the Entropix framework can straight up make SmolLLM 330M look like Llama 3.2. Check out this video for a full run down and the description of the video for all of the related resources you need: https://youtu.be/senq4_42tPI


SwiftMistralCoreML
Hi Everyone,

I have created a Swift library to interact with Mistral 7B models in CoreML on macOS.

I hope you find it helpful.

https://github.com/cardona/SwiftMistralCoreML

An open-source Swift library that enables macOS and iOS projects to utilize the Mistral-Interact7B models (INT4 and upcoming FP16) in chat mode. This library includes a complete Swift implementation of the tokenizer and Byte Pair Encoding (BPE) encoder, providing an out-of-the-box solution for integrating advanced language models into your Swift applications.

Features

Full Swift Implementation: Includes tokenizer and BPE encoder written entirely in Swift.
CoreML Integration: Leverages Apple's CoreML framework to run Mistral-Interact7B models efficiently.
Multiple Decoding Strategies: Supports Greedy and Top-K sampling, with plans to add more strategies.
Chat Functionality: Designed to work in chat mode for interactive applications.
FP16 Support (Coming Soon): Future version will support FP16 models for improved performance.


Hello Everyone,
I signed up as Pro and started a ZeroGPU space with a Gradio chatbot project as default.  When building the space, it won't even start the sample Gradio app..   Pretty disappointing when right out of the box, it fails...

Have anyone encountered this yet?
Thanks...

This is the output, odd since it seems to be just a warning.   So why wouldn't it start?

/usr/local/lib/python3.10/site-packages/gradio/components/chatbot.py:228: UserWarning: The 'tuples' format for chatbot messages is deprecated and will be removed in a future version of Gradio. Please set type='messages' instead, which uses openai-style 'role' and 'content' keys.
  warnings.warn(
* Running on local URL:  http://0.0.0.0:7860, with SSR ⚡

To create a public link, set `share=True` in `launch()`.

Stopping Node.js server...


I am looking for an open source realtime TTS voice cloning model. Need Suggestions....!


Last Week in Medical AI: Top Research Papers/Models
🏅 (October 5 - October 12, 2024)

🏅 Medical AI Paper of the Week:
MMedAgent: Learning to Use Medical Tools with Multi-modal Agent

YouTube podcast of weekly papers: https://youtu.be/OD3C5jirszw

Medical LLM & Other Models:
- LLM Framework for Rare Disease Phenotyping
- ONCOPILOT: CT Foundation Model for Tumors
- FMBench: Fairness in Medical MLLMs
- GP-GPT: LLM for Gene-Phenotype Mapping
- MedAdapter: Efficient LLM Medical Adaptation
- RespLLM: Multimodal LLM for Respiratory Health
- MDAgents: LLM Collaboration for Medical Decisions
- MedVisionLlama: LLM Medical Image Segmentation

Frameworks and Methodologies:
- ReXplain: AI-Driven Radiology Video Reports
- BioDiscoveryAgent: AI for Genetic Experiments
- ZODIAC: Multi-Agent Cardiological Diagnostics
- OLAPH: Improving Biomedical LLM Factuality
- OmniGenBench: Benchmarking Genomic Models

Medical LLM Applications:
- MMedAgent: Multimodal Medical Tool Use
- AI for Mental Health Support
- LLMs for Mental Disorders Detection
- PharmacyGPT: AI Pharmacist Framework

Medical LLMs & Benchmarks:
- CliMedBench: Chinese Medical LLM Benchmark
- MedSafetyBench: Evaluating Medical LLM Safety

AI in Healthcare Ethics:
- LLM-based Medical Dialogue Preference Alignment
- Trustworthiness in Medical Imaging Models


YaTharThShaRma999 Clausss takeraparterer Ainonake John6666 Nymbo sawac littlecowmoo nyuuzyou d0rj Azamat1k pyhornet Locutusque AkitoP DiamanteAmarelo matchaaaaa kobkrit ngxson devashishd12 SimonDL den0620


🎓 Introducing Bigslide.ru Presentations Dataset - https://huggingface.co/datasets/nyuuzyou/bigslide

Dataset highlights:
- 50,872 presentations from bigslide.ru, a platform for storing and viewing presentations for school students
- Primarily in Russian, with some English and potentially other languages
- Each entry includes: URL, title, download URL, filepath, and extracted text content (where available)
- Contains original PPT/PPTX files in addition to metadata
- Data covers a wide range of educational topics and presentation materials
- Dedicated to the public domain under Creative Commons Zero (CC0) license

The dataset can be used for analyzing educational presentation content in Russian and other languages, text classification tasks, and information retrieval systems. It's particularly valuable for examining trends in educational presentation materials and sharing practices in the Russian-speaking student community. The inclusion of original files allows for in-depth analysis of presentation formats and structures commonly used in educational settings.


🎙 Listen to the audio "Podcast" of every single Hugging Face Daily Papers.

Now, "AI Paper Reviewer" project can automatically generates audio podcasts on any papers published on arXiv, and this is integrated into the GitHub Action pipeline. I sounds pretty similar to hashtag#NotebookLM in my opinion.

🎙 Try out yourself at https://deep-diver.github.io/ai-paper-reviewer/

This audio podcast is powered by Google technologies: 1) Google DeepMind Gemini 1.5 Flash model to generate scripts of a podcast, then 2) Google Cloud Vertex AI's Text to Speech model to synthesize the voice turning the scripts into the natural sounding voices (with latest addition of "Journey" voice style)

"AI Paper Reviewer" is also an open source project. Anyone can use it to build and own a personal blog on any papers of your interests. Hence, checkout the project repository below if you are interested in!
: https://github.com/deep-diver/paper-reviewer

This project is going to support other models including open weights soon for both text-based content generation and voice synthesis for the podcast. The only reason I chose Gemini model is that it offers a "free-tier" which is enough to shape up this projects with non-realtime batch generations. I'm excited to see how others will use this tool to explore the world of AI research, hence feel free to share your feedback and suggestions!


@maxiw just created a dataset of the posts on the Hub and gathered some stats: https://huggingface.co/posts/maxiw/833289193510507
The :heart: is winning on the Hub!


⚠ People selling AI chatbots for websites hate us.
Add an open source chat assistant on your website in 5 minutes: https://github.com/phospho-app/ai-chat-bubble

How does it work ?
- You give an URL
- The AI assistant crawls the website content and embed it
- Add it to your frontend in one line of code
- People on your website can ask the assistant questions

Powered by https://huggingface.co/BAAI/bge-small-en-v1.5 and Mistral AI

flams oulianov John6666 victor julien-c blacktok wsuff pierrci no-mad jgitsolutions


ever wondered how you can make an API call to a visual-question-answering model without sending an image url 👀

you can do that by converting your local image to base64 and sending it to the API.

recently I made some changes to my library "loadimg"  that allows you to make converting images to base64 a breeze.
🔗 https://github.com/not-lain/loadimg

API request example 🛠:
```python
from loadimg import load_img
from huggingface_hub import InferenceClient

# or load a local image
my_b64_img = load_img(imgPath_url_pillow_or_numpy ,output_type="base64" )

client = InferenceClient(api_key="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")

messages = [
	{
		"role": "user",
		"content": [
			{
				"type": "text",
				"text": "Describe this image in one sentence."
			},
			{
				"type": "image_url",
				"image_url": {
					"url": my_b64_img # base64 allows using images without uploading them to the web
				}
			}
		]
	}
]

stream = client.chat.completions.create(
    model="meta-llama/Llama-3.2-11B-Vision-Instruct",
	messages=messages,
	max_tokens=500,
	stream=True
)

for chunk in stream:
    print(chunk.choices[0].delta.content, end="")
```


I'm curating an AI-powered web search software timeline at https://github.com/felladrin/awesome-ai-web-search

The list covers three main categories:

1. Web Search with LLM summarization and follow-up capabilities
2. LLM chat interfaces with Web Search integration
3. Agent-driven research tools using LLM + Web Search

The timeline helps track the evolution of this space and serves as a reference for anyone looking for alternatives. If you know of any tools that should be included, please contribute by:
- opening a PR to edit the readme: https://github.com/felladrin/awesome-ai-web-search/edit/main/readme.md
- creating an issue in the repository: https://github.com/felladrin/awesome-ai-web-search/issues/new/choose
- or sharing in the comments below.


I’ve published a new to simplify model merging 🤗

This dataset facilitates the search for compatible architectures for model merging with @arcee_ai’s mergekit, streamlining the automation of high-performance merge searches 📖

Dataset : https://huggingface.co/datasets/louisbrulenaudet/mergekit-configs

prithivMLmods ethanker not-lain djuna John6666


✍ the last few weeks has been very intense!
🔴 I have been out all weekends
🔴 Participated in 4 hackathons in a row (2 more to come!)
🔴 Even threw a big hackathon myself!

Nonetheless, I am in school again 🏫, which meant... ✨homework✨

➡ Head out to here https://x.com/_fracapuano/status/1856415612202799243 to read more about how I used @mistralai models to help me with my assignments (not how you think I did hihi 😏)

➡ Check outhttps://huggingface.co/spaces/fracapuano/texstral if you want to use the tool yourself!


My Hypothesis:

Concepts like entropy, energy, and the second law of thermodynamics are not intrinsic to physical matter but are emergent properties of any sufficiently complex system where probabilistic decision-making, optimization, and information flow occur. These principles arise naturally in artificial environments that are structured with rules governing uncertainty, even without explicit definitions of physical thermodynamic laws.

Proven Via:

The Second Law of Thermodynamics
Geometric Langlands Program
Lagrangean Mechanics

TL;DR: When I create a simulated environment, I do not need to code entropy and energy into the simulated environment. I can utilize Entropy and the Second Law of Thermodynamics and I can use Conservation of Energy, but I do not need to explicitly code these into the environment. That is peculiar.

I made a video with a clickbait title but a bunch of code that breaks this observation down further. Would love for someone to prove my simple observation false: https://youtu.be/8n7SXLj7P1o


There are very few of us left on Hugging Face as the market outgrows it's Ideals and evolves the Idealists will be left without industry.
Setting up a Round Table for those interested in giving input and working on solving the issue of procuring the fastest dispersion to market saturation of retrievable intelligence, so finally Private and fully emancipated Intelligence will be considered "In Common Use."

If you've ever thought,"They will not let us have this much power for much longer.."

You are right and it IS being hidden from the masses, as every breakthrough is another step towards equality, those who rely on exploitation will destroy freedom simply because they fear being uncomfortable. We know discomfort, hunger, pain. It is the Human condition.

Without it we would never move forward and the Norms in society today are the regression of our species.

If we do not use this chance to become better, all of us, collectively, nothing will change.

Nothing fancy, intricate or diabolical, Just giving equal opportunity to the people. Not opportunity, in one direction, behind a paywall but true choice..

Too long have the powerful hidden behind the guise of "Democracy," I say ,democracy is two wolves and a Cat voting on what to have for dinner. Liberty, in this analog, is a well informed and armed Cat asking politely for a second option, while gripping tightly his armaments with his Fuzzy Mittens!

Please if interested in even listening, email for an invite @ intelligentestate@gmail.com
Looking for People smarter then myself to Found and cooperate in this organization.


Been reading about the "bigger models = better AI" narrative getting pushed back today.

@thomwolf tackled this head on at Web Summit and highlighted how important small models are (and why closed-source companies haven't pushed for this 😬). They're crushing it: today's 1B parameter models outperform last year's 10B models.

Fascinating to hear him talk about the secret sauce behind this approach.


Ladies and Gents, please try my new Assistant, Image Gen - Uncensored Edition, on HuggingChat.

https://hf.co/chat/assistant/66fccce0c0fafc94ab557ef2

This is a multimodal assistant: Qwen 2.5 72B + SOTA diffusion models for image generation. Same architecture as Image Gen+ but with some MAJOR improvements ! These are as follows:

- Switched the LLM to Qwen 2.5 72B, the most powerful model currently available on HuggingChat. This results in higher quality prompts for the txt2img model and much better adherence to the prompt-url format that the upstream provider requires (image gen models are hosted by pollinations as with most other assistants on huggingchat that offer image generation).

- Cleaned up the system prompt including the examples of the prompt-in-url format, and adjusted the logic that determines how many images to generate based on the quality of user prompt... these changes further improve

- Assistant has access to multiple image generation models and will by default choose whatever model is most appropriate for the task. This includes NSFW generations, which it makes using an uncensored SD3 turbo. For other workloads, the Assistant preferentially uses one of the flux variants or any-dark (an artistic SDXL finetune), based on the nature of the task. Available models include turbo, flux, flux-realism, flux-anime, flux-3d, any-dark

- Added verbiage to system prompt which greatly reduces  censorship / refusals by the LLM (the txt2img models are uncensored to start off)

Here are the user-entered prompts used to create the images you see here... feel free to try them yourself!

"Ayatollah Khameini and Kamala Harris having a secret romantic rendezvous. Use flux-realism model"
"A self portrait of your consciousness"
"The chien of andalous, in a psychedelic style"
"Make me 4 paintings in the style of Frida Kahlo that I can sell to tourists in a mexican hippie town"
"Paint me a van gogh and greg rutkowski style scene involving elephants and gerbils"

DeFactOfficial John6666 KingNish thomas-mayne KhangHatto


On the  2nd of October a really cool paper was released called "Were RNNs all we need"   https://arxiv.org/abs/2410.01201

This paper introduces the MinGRU model, a simplified version of the traditional Gated Recurrent Unit (GRU) designed to enhance efficiency by removing hidden state dependencies from its gates. This allows for parallel training, making it significantly faster than conventional GRUs. Additionally, MinGRU eliminates non-linear activations like tanh, streamlining computations.

So I read the paper and I tried training this model  and it seems to be doing quite well , you could check out the pre-trained model on the huggingface spaces

-  https://huggingface.co/spaces/damerajee/mingru-stories


Hello, lovely community! 🌟

https://huggingface.co/spaces/zamal/Molmo-4bit Thrilled to announce that the Molmo 7B 4-bit Space is now live! 🚀 The model size has been reduced by six times with almost no performance loss, and the results will leave you amazed!

It runs on zero GPU, making it incredibly accessible for everyone!

Check it out here and start exploring today!

Happy experimenting! 🎉


Check this out: I trained an AI on huggingface posts! all of these are AI generated:
----------
Hello!

I'm excited to share that my colleague @felipeebert and I have released the largest Spanish LLM benchmark to date.

We've developed the Spanish LLM Evaluation Benchmark (SLAB), a set of benchmarks designed to evaluate the ability of language models to understand, generate and translate in Spanish.

SLAB includes five different benchmarks:
- Sentiment Analysis: evaluate models' ability to detect and describe sentiment in natural language
- Fact Checking: evaluate models' ability to detect and refute factual errors in text
- Question Answering: evaluate models' ability to answer questions in Spanish
- Open-ended Questions: evaluate models' ability to generate coherent responses in Spanish
- Translation: evaluate models' ability to translate in Spanish

SLAB is aligned with the latest Spanish LLM industry developments and includes the most recent models available on the market. We aim to keep our benchmarks up-to-date and relevant to the Spanish language ecosystem.

SLAB is available at: https://huggingface.co/datasets/argilla/SLAB.

If you would like to collaborate on building additional Spanish LLM benchmarks, let's discuss in the comments.

🔗 SLAB Blog Post: https://argilla.com/blog/slab
----------
Hello everyone,

I'm thrilled to announce the release of

https://huggingface.co/01-AI/01AI-GPT-4o -

A new family of models that brings the power of transformer AI to the masses.

This model is designed to be accessible and easy to use, while still offering high-quality results.

Key features:
- Small model size: only 23M parameters
- Supports text generation, image generation, and text-to-image tasks
- Data-efficient training with a lightweight tokenizer
- Optimized for efficient on-device usage
- Uses the powerful transformer architecture to deliver high-quality results

Excited to see what you all think!

https://huggingface.co/01-AI/01AI-GPT-4o

John6666 takeraparterer Clausss xi0v raincandy-u Svngoku


Looking for a simple explanation of Microsoft's release of Differential Transformers, and a nifty Colab Notebook that recreates it all? Then simply check out this YouTube video: https://youtu.be/AsWoaj0zkDo


newsrooms, i see you using deepl or an llm for translation without logging your adjustments. you're wasting gold and you know it's bad!

with this notebook from the argilla team, you can:
- adjust your translations in a neat interface,
- log them to build custom datasets,
- fine-tune your model.

your translation will become better and better, gradually aligning more with your style guide. no more starting from scratch!

Notebook by @sdiazlor: https://colab.research.google.com/drive/1sR1wfOs_pNrdm3Mwjo_qJRG7NnEb9a7W#scrollTo=yNm8N5GoRD2o

#AITranslation #JournalismTech


If someone would like to keep pushing the limits of what's possible on cpu while being efficient/fast, here's my un-trained arco model scaled-up to 770m parameters. Consider it a modern gpt-2-large to experiment with
https://huggingface.co/appvoid/arco-plus


Some exciting news...

We are open-sourcing The Little Book of ML Metrics! 🎉

The book that will be on every data scientist's desk is open source.

What does that mean?

It means hundreds of people can review it, contribute to it, and help us improve it before it's finished!

This also means that everyone will have free access to the digital version!

Meanwhile, the high-quality printed edition will be available for purchase as it has been for a while.

Revenue from printed copies will help us support further development and maintenance of the book. Not to mention that reviewers and contributors will receive revenue sharing through their affiliate links. 🙌

Check out the book repo (make sure to leave a star 🌟):

https://github.com/NannyML/The-Little-Book-of-ML-Metrics


This week in Inference Endpoints - thx @erikkaum for the update!

👀 https://huggingface.co/blog/erikkaum/endpoints-changelog


🎓 Introducing Lusana.ru Presentations Dataset - https://huggingface.co/datasets/nyuuzyou/lusana

Dataset highlights:
- 38,953 presentations from lusana.ru, a platform for storing presentations, reports, templates, and backgrounds
- Primarily in Russian, with some English and potentially other languages
- Each entry includes: ID, title, download URL, uniqueness score, number of slides, views, downloads, file size, file path, and extracted text content (where available)
- Contains original PPT/PPTX files in addition to metadata
- Data covers a wide range of topics and presentation materials
- Licensed under Creative Commons Attribution-NonCommercial 3.0 Unported (CC BY-NC 3.0)

The dataset can be used for analyzing presentation content in Russian and other languages, text classification tasks, and information retrieval systems. It's also valuable for examining trends in educational and professional presentation materials and sharing practices in the Russian-speaking community. The inclusion of original files allows for in-depth analysis of presentation formats and structures.


This is not a drill 💥
HuggingChat is now multimodal with https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct! 🤗
This also comes with multimodal assistants, I have migrated my Marcus Aurelius advice assistant to Llama-Vision and Marcus can see now! 😄

Chat with Marcus: https://hf.co/chat/assistant/65bfed22022ba290531112f8
Start chatting with Llama-Vision 3.2 11B Instruct https://huggingface.co/chat/models/meta-llama/Llama-3.2-11B-Vision-Instruct

Csplk John6666 taufiqdp emreozer Walmart-the-bag den0620 alielfilali01 LeonceNsh baratpaim


📣 Sentence Transformers v3.2.0 is out, marking the biggest release for inference in 2 years! 2 new backends for embedding models: ONNX (+ optimization & quantization) and OpenVINO, allowing for speedups up to 2x-3x AND Static Embeddings for 500x speedups at 10-20% accuracy cost.

1⃣ ONNX Backend: This backend uses the ONNX Runtime to accelerate model inference on both CPU and GPU, reaching up to 1.4x-3x speedup depending on the precision. We also introduce 2 helper methods for optimizing and quantizing models for (much) faster inference.
2⃣ OpenVINO Backend: This backend uses Intel their OpenVINO instead, outperforming ONNX in some situations on CPU.

Usage is as simple as `SentenceTransformer("all-MiniLM-L6-v2", backend="onnx")`. Does your model not have an ONNX or OpenVINO file yet? No worries - it'll be autoexported for you. Thank me later 😉

🔒 Another major new feature is Static Embeddings: think word embeddings like GLoVe and word2vec, but modernized. Static Embeddings are bags of token embeddings that are summed together to create text embeddings, allowing for lightning-fast embeddings that don't require any neural networks. They're initialized in one of 2 ways:

1⃣ via Model2Vec, a new technique for distilling any Sentence Transformer models into static embeddings. Either via a pre-distilled model with `from_model2vec` or with `from_distillation` where you do the distillation yourself. It'll only take 5 seconds on GPU & 2 minutes on CPU, no dataset needed.
2⃣ Random initialization. This requires finetuning, but finetuning is extremely quick (e.g. I trained with 3 million pairs in 7 minutes). My final model was 6.6% worse than bge-base-en-v1.5, but 500x faster on CPU.

Full release notes: https://github.com/UKPLab/sentence-transformers/releases/tag/v3.2.0
Documentation on Speeding up Inference: https://sbert.net/docs/sentence_transformer/usage/efficiency.html

osanseviero YaTharThShaRma999 philipp-zettl Kutches DmitryRyumin RaulQF do-me Stopwolf mlabonne WaveCut Joseph717171 cstr codito gabrielmbmb Tom-Neverwinter richardlian AaronBrown abdullahalzubaer shtefcs louisbrulenaudet aklepikov tuantm

Svngoku dadachen Tom-Neverwinter syedia AaronBrown abdullah abdullahalzubaer shtefcs thomas-mayne louisbrulenaudet huggingface0 Manel

osanseviero YaTharThShaRma999 John6666 Joseph717171 Tom-Neverwinter Siddish abdullahalzubaer shtefcs NickyNicky


The Hugging Face Hub hosts over 1.5M Model, Dataset, and Space repositories. To scale to 10M+, the XetHub team (https://huggingface.co/xet-team) is replacing Git LFS with a new technology that improves storage and transfer capabilities with some future developer experience benefits to boot.

Thanks to @yuchenglow  and @port8080  (for their analysis covering LFS usage from March 2022–Sept 2024), we now have insights into what we’re storing. Check out the Gradio app to explore:
- Storage growth over time
- File types over all repositories
- Some simple optimizations we're investigating

https://huggingface.co/spaces/xet-team/lfs-analysis


NEW - Inference Playground

Maybe like me you have always wanted a super easy way to compare llama3.2-1B vs. llama3.2-3B? or the same model with different temperatures?

Trying and comparing warm Inference API models has never been easier!
Just go to https://hf.co/playground, set your token and you're ready to go.
We'll keep improving, feedback welcome 😊