Good folks at @Apple have developed a novel method called KV Prediction that significantly reduces the "time to first token" (TTFT) for on-device LLM inference.

Some highlights of the paper:

• Uses a small auxiliary transformer model to efficiently predict the KV cache of a larger base model
• Reduces TTFT by up to 4x while retaining 60-80% accuracy on benchmarks
• Achieves Pareto-optimal efficiency-accuracy trade-off compared to baselines
• Demonstrates 15-50% relative accuracy improvements on TriviaQA at equal TTFT FLOP budgets
• Shows up to 30% accuracy gains on HumanEval code completion at fixed TTFT FLOP counts
• Validated on Apple M2 Pro CPU, proving FLOP gains translate to real-world speedups

So, how's it done?

Based on the KV Prediction method described in the paper, here are the key steps for how it's done:

1. Choose a base model and an auxiliary model:
- The base model is a larger, pretrained transformer model that will be used for final generation.
- The auxiliary model is a smaller transformer model used to efficiently process the input prompt.

2. Design the KV predictor:
- Create a set of learned linear projections to map from the auxiliary model's KV cache to the base model's KV cache.
- Define a mapping from auxiliary cache layers to base cache layers.

3. Training process:
- Pass input tokens through the auxiliary model to get its KV cache.
- Use the KV predictor to generate a predicted KV cache for the base model.
- Run the base model using the predicted KV cache and compute losses.
- Backpropagate errors through the frozen base model to update the auxiliary model and KV predictor.

4. Inference process:
- Process the input prompt with the auxiliary model to get its KV cache.
- Use the KV predictor to generate the predicted base model KV cache.
- Run a single token generation step with the base model using the predicted KV cache.
- Continue autoregressive generation with the base model as normal.

Excited to hear your thoughts!

John6666 YaTharThShaRma999 barakplasma KingNish myronkoch AtAndDev GoDjMike


A new `timm` release (1.0.11) is out now. A also wrote an article on one of the included models: https://huggingface.co/blog/rwightman/mambaout

Featured in the release are:
  * The MambaOut model, a cheeky arch inspired by SSM but without the SSM part, a ConvNeXt with gating.
  * Several timm trained MambaOut variations with arch tweaks and ImageNet-12k pretrain to verify scaling, supplement ported weights.
  * The smallest MobileNetV4, a 0.5x width scaled Conv-Small.
  * Two impressive MobileNetV3 Large models outperforming all previous, using MNV4 Small recipe.
  * 'Zepto,' a new compact ConvNeXt variant even smaller than the previous Atto, 2.2M params, RMSNorm, and solid results for its size.
  * Newly ported SigLIP SO400M/16 ViT multi-lingual weights, the largest i18n weights, prevous was B/16.
  * Two ImageNet-1k fine-tuned SigLIP SO400M models at 378x378
  * InternViT 300M weight port. A really solid ViT encoder distilled from OpenGVLab 6B VL model encoder.
  * An assortment of very small, sub 1M param pretrained test models to improve library unit tests and serve low-resource applications.


InstructHumans: Editing Animated 3D Human Textures with Instructions (2404.04037)

🚀💃🌟 New Research Alert (Avatars Collection)! 🌟🕺🚀
📄 Title: InstructHumans: Editing Animated 3D Human Textures with Instructions

📝 Description: InstructHumans is a novel framework for text-instructed editing of 3D human textures that employs a modified Score Distillation Sampling (SDS-E) method along with spatial smoothness regularization and gradient-based viewpoint sampling to achieve high-quality, consistent, and instruction-true edits.

👥 Authors: Jiayin Zhu, Linlin Yang, Angela Yao

🔗 Paper: https://huggingface.co/papers/2404.04037

🌐 Web Page: https://jyzhu.top/instruct-humans
📁 Repository: https://github.com/viridityzhu/InstructHumans

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #InstructHumans #3DTextureEditing #TextInstructions #ScoreDistillationSampling #SDS-E #SpatialSmoothnessRegularization  #3DEditing #AvatarEditing #DeepLearning #Innovation


We just released gradio version 4.26.0 ! We *highly* recommend you upgrade your apps to this version to bring in these nice changes:

🎥 Introducing the API recorder. Any gradio app running 4.26.0 and above will have an "API Recorder" that will record your interactions with the app and auto-generate the corresponding python or js code needed to recreate those actions programmatically. It's very neat!

📝 Enhanced markdown rendering in gr.Chatbot

🐢 Fix for slow load times on spaces as well as the UI locking up on rapid generations

See the full changelog of goodies here: https://www.gradio.app/changelog#4-26-0

samusenps DmitryRyumin dillfrescott gsarti osanseviero aceeee ysharma on1onmangoes


Fun fact about evaluation, part 2!

How much do scores change depending on prompt format choice?

Using different prompts (all present in the literature, from `Prompt question?` to `Question: prompt question?\nChoices: enumeration of all choices\nAnswer: `), we get a score range of...

10 points for a single model!
Keep in mind that we only changed the prompt, not the evaluation subsets, etc.
Again, this confirms that evaluation results reported without their details are basically bullshit.

Prompt format on the x axis, all these evals look at the logprob of either "choice A/choice B..." or "A/B...".

Incidentally, it also changes model rankings - so a "best" model might only be best on one type of prompt...

eduagarcia SivilTaram andysalerno samusenps osanseviero EricNarro


text-generation-inference (TGI) is now fully open-source again!

Along with text-embeddings-inference.

We just switched both of those repos' license back to Apache 2. 🔥

clefourrier Saugatkafley samusenps jcudit flozi00 osanseviero joennlae Joseph717171 clem freddyaboulton

sugatoray gsarti kramp osanseviero Joseph717171 clem timestop1

Sylvestre clefourrier ehcalabres osanseviero Joseph717171 clem


No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency
  Determines Multimodal Model Performance (2404.04125)

No "Zero-Shot" Without Exponential Data

Pretraining Concept Frequency Determines Multimodal Model Performance

https://huggingface.co/papers/2404.04125

Web-crawled pretraining datasets underlie the impressive "zero-shot" evaluation performance of multimodal models, such as CLIP for classification/retrieval and Stable-Diffusion for image generation. However, it is unclear how meaningful the notion of "zero-shot" generalization is for such multimodal models, as it is not known to what extent their pretraining datasets encompass the downstream concepts targeted for during "zero-shot" evaluation. In this work, we ask: How is the performance of multimodal models on downstream concepts influenced by the frequency of these concepts in their pretraining datasets? We comprehensively investigate this question across 34 models and five standard pretraining datasets (CC-3M, CC-12M, YFCC-15M, LAION-400M, LAION-Aesthetics), generating over 300GB of data artifacts. We consistently find that, far from exhibiting "zero-shot" generalization, multimodal models require exponentially more data to achieve linear improvements in downstream "zero-shot" performance, following a sample inefficient log-linear scaling trend. This trend persists even when controlling for sample-level similarity between pretraining and downstream datasets, and testing on purely synthetic data distributions. Furthermore, upon benchmarking models on long-tailed data sampled based on our analysis, we demonstrate that multimodal models across the board perform poorly. We contribute this long-tail test set as the "Let it Wag!" benchmark to further research in this direction. Taken together, our study reveals an exponential need for training data which implies that the key to "zero-shot" generalization capabilities under large-scale training paradigms remains to be found.

samusenps vicgalle vladbogo adhirajghosh KvrParaskevi


✨ Easy Synthetic Dataset File Generation using LLM DataGen ! Link: https://huggingface.co/spaces/lhoestq/LLM_DataGen

features + how it works:

✍ Generate the dataset content you want just by entering a file name
💡 Optionally specify the column names you need
💨 The dataset is streamed and generated on-the-fly in JSON Lines format
✅ Generation is constrained to always output valid JSON

How does this work ?
1/ Enter a file name
2/ The model generates column names for such a file. Using structured generation, it can generate 2 to 5 column names using lower case characters and underscores. I use a prompt that asks to generate column names for a realistic dataset and low temperature.
3/ The columns are used to update the Finite State Machine for the dataset content structured generation, so that it is used to generate JSON objects using those columns
4/ The model generates JSON objects using structured generation again, using the updated Finite State Machine. I use a prompt that asks for realistic data and a temperature of 1.

> Why update a Finite State Machine instead of re-creating one ?

Creating one can take up to 30sec, while updating one takes 0.1s (though it requires to manipulate a graph which is not easy to implement)

> Batched generation is faster, why not use it ?

Generate in batches is faster but tends to generate duplicates for this demo.
Further work can be to provide different prompts (one per sequence in the batch) to end up with a different distribution of sequences in each batch. Or implement a custom sampler that would forbid generating the same data in sequences of the same batch.

> How does structured generation work ?

I used the `outlines` library with `transformers` to to define a JSON schema that the generation has to follow. It uses a Finite State Machine with `token_id` as transitions.

Let me know what you think ! And feel free to duplicate/modify it to try other models/prompts or sampling methods :)


Brain2Music: Reconstructing Music from Human Brain Activity (2307.11078)

Decoding speech from non-invasive brain recordings (2208.12266)

Seeing through the Brain: Image Reconstruction of Visual Perception from
  Human Brain Signals (2308.02510)

DreamDiffusion: Generating High-Quality Images from Brain EEG Signals (2306.16934)

NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual
  Pretraining and Multi-level Modulation (2403.18211)

NeuroCine: Decoding Vivid Video Sequences from Human Brain Activties (2402.01590)

Cinematic Mindscapes: High-quality Video Reconstruction from Brain
  Activity (2305.11675)

MinD-3D: Reconstruct High-quality 3D objects in Human Brain (2312.07485)

DreamGaussian4D: Generative 4D Gaussian Splatting (2312.17142)

Sora: A Review on Background, Technology, Limitations, and Opportunities
  of Large Vision Models (2402.17177)

Hello world!

I'd like to share with you all today some specific Research about the Brain & some surface level thoughts

Music (Frequencies)
https://huggingface.co/papers/2307.11078

Speech
https://huggingface.co/papers/2208.12266

Image
https://huggingface.co/papers/2308.02510
https://huggingface.co/papers/2306.16934
https://huggingface.co/papers/2403.18211

Video
https://huggingface.co/papers/2402.01590
https://huggingface.co/papers/2305.11675

3D
https://huggingface.co/papers/2312.07485

Potential Opportunities For BCI

4D
https://huggingface.co/papers/2312.17142

Realistic High Quality Video
https://huggingface.co/papers/2402.17177

https://huggingface.co/collections/samusenps/bci-661206b642da659656474db2

Reading minds is cool & useful but could be utilized for many things other than thought interrogation 👀

Current Co pilots are Boring! & have much untapped potential & also people don't seem to want autonomous agents replacing them although it is inevitable for some cases

I believe if humans want to become an interplanetary species that can utilize our accumulative research we need to extend our brain with technology in order to be smarter. Imagine a Co-pilot for your head, Adding extra ‘RAM’ to the brain, or even Processing external data within the brain.

Ok people are afraid of implanting computer chips within their brains, what if someone hacks it ? , the invasive possibilities are crazy!

How can we ensure Safety & Interpretability in brain computer interfaces
1. External Non Invasive Brain Computer interfaces [ think similar to https://neurosity.co/crown (overpriced IMO & Hardware is closed source proprietary, who knows what they’re doing 👁) ]
2. Full Reproducible Open-Source Stack Brain computer interface down to the hardware, operating system, and application level.
3. Maybe you can't, there may always be a risk of danger, though not as consequential with 1 & 2

samusenps INMYEYES DmitryRyumin Jigya gsarti diwank osanseviero lunarflu victor medmac01 not-lain s3nh efecelik

digiplay osanseviero flyineye lunarflu medmac01 not-lain s3nh


3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (2312.09228)

🚀🕺🌟 New Research Alert - CVPR 2024 (Avatars Collection)! 🌟💃🚀
📄 Title: 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting 🔝

📝 Description: 3DGS-Avatar is a novel method for creating animatable human avatars from monocular videos using 3D Gaussian Splatting (3DGS). By using a non-rigid deformation network and as-isometric-as-possible regularizations, the method achieves comparable or better performance than SOTA methods while being 400x faster in training and 250x faster in inference, allowing real-time rendering at 50+ FPS.

👥 Authors: Zhiyin Qian, Shaofei Wang, Marko Mihajlovic, Andreas Geiger, Siyu Tang

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2312.09228

🌐 Github Page: https://neuralbodies.github.io/3DGS-Avatar/
📁 Repository: https://github.com/mikeqzy/3dgs-avatar-release

📺 Video: https://www.youtube.com/watch?v=FJ29U9OkmmU

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #3DGSAvatar #3DAvatar #3DGaussianSplatting #AnimatableAvatars #MonocularVideo #RealTimeRendering #FastTraining #EfficientInference #CVPR2024 #DeepLearning #Innovation

DmitryRyumin AlekseiPravdin Jigya YoungMetal yehiaserag dillfrescott osanseviero Lei-bw Redcalx34B mariasandu lognat0704

samusenps AlekseiPravdin Jigya YoungMetal dillfrescott osanseviero Lei-bw Redcalx34B lognat0704

DmitryRyumin Jigya YoungMetal dillfrescott


ReFT: Representation Finetuning for Language Models (2404.03592)

🔍 Today's pick in Interpretability & Analysis of LMs: ReFT: Representation Finetuning for Language Models by @zhengxuanzenwu @aryaman Z. Wang @atticusg D. Jurafsky @manning @cgpotts

This work introduces Representation fine-tuning (ReFT), a framework using learned inference-time interventions as efficient yet effective alternatives to PEFT weight adaptation. LoReFT, a ReFT variant intervening linearly on a representation subspaces, is evaluated against several PEFT approaches showing SOTA performances across popular benchmark with 10-50x speedup. The 🤗-compatible pyreft library is introduced to simplify ReFT usage.

This is one of the most convincing practical applications of interpretability methods/insights I've seen in recent years, and I'm looking forward to people combining this with methods to disentangle features like SAEs and Backpack LMs for making interventions more interpretable!

📄 Paper: https://huggingface.co/papers/2404.03592

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9

samusenps aryaman Jigya zhengxuanzenwu diwank srisree


To date, louisbrulenaudet/Maxine-34B-stock is the "Best 🤝 base merges and moerges model of around 30B" on the Open LLM Leaderboard ❤🔥

It is a practical application of the stock method recently implemented by @arcee-ai in the MergeKit :
```yaml
models:
    - model: ConvexAI/Luminex-34B-v0.2
    - model: fblgit/UNA-34BeagleSimpleMath-32K-v1
merge_method: model_stock
base_model: abacusai/Smaug-34B-v0.1
dtype: bfloat16
```
Model : https://huggingface.co/louisbrulenaudet/Maxine-34B-stock
LLM Leaderboard best models ❤🔥 Collection : https://huggingface.co/collections/open-llm-leaderboard/llm-leaderboard-best-models-652d6c7965a4619fb5c27a03

samusenps dhjang ben-wise Jigya osanseviero not-lain


Releasing H2O-Danube2-1.8b - a new and improved language model being the current top model on the Open LLM Leaderboard.

The model is trained on additional 2T tokens (total of 3T tokens) with various data mix stages as a result of extensive experimentation. Focus is on natural language tasks.

Base weights: https://huggingface.co/h2oai/h2o-danube2-1.8b-base
SFT weights: https://huggingface.co/h2oai/h2o-danube2-1.8b-sft
Chat weights: https://huggingface.co/h2oai/h2o-danube2-1.8b-chat


Honestly i don't understand how come we as the open source community haven't surpassed GPT-4 yet ? Like for me it looks like everything is out there just need to be exploited! Clearly specialized small models outperforms gpt4 on downstream tasks ! So why haven't we just trained a 1B-2B really strong general model and then continue pertained and/or finetuned it on datasets for downstream tasks like math, code...well structured as Textbooks format or other datasets formats that have been proven to be really efficient and good! Ounce you have 100 finetuned model, just wrap them all into a FrankenMoE and Voila ✨
And that's just what a NOOB like myself had in mind, I'm sure there is better, more efficient ways to do it ! So the question again, why we haven't yet ? I feel I'm missing something... Right?

samusenps awhsnap victor m18coppola visitthething


Very interesting model just released by MyShell: https://huggingface.co/jetmoe/jetmoe-8b . It's a 8B-parameters MoE LLM so 2.2B active parameters, really efficient.

Main characteristics:
- impressive performances for its size (beating https://huggingface.co/meta-llama/Llama-2-7b and https://huggingface.co/huggyllama/llama-13b)
- combine Mixture of Attention heads (MoA) and Mixture of MLP Experts (MoE) – 8 experts with 2 being active for each token
- trained on a rather limited 1.25T tokens from publicly available datasets – training recipe follows the MiniCPM's two-phases training method => first time I see this for a 2B+ model
- $100k to train
- open weights - open sharing of recipes - open dataset - open code => ♡
- still interesting room to improve performances (be it only by training longer)

Links:
- report: https://research.myshell.ai/jetmoe
- model: https://huggingface.co/jetmoe/jetmoe-8b
- code: https://github.com/myshell-ai/JetMoE

Note: I actually detailed all of the MiniCPM schedule, Mixture-of-expert (MoE) and many of the datasets used in this work in my recent little guide to building LLMs in 2024, so feel free to check it out if you want to learn more on these topics: https://www.youtube.com/watch?v=2-SPH9hIKT8

samusenps a9i giux78 dillfrescott clem osanseviero

samusenps ajibawa-2023 dillfrescott clem osanseviero


🐦 The IBIS Challenge: an open competition in Inferring and predicting transcription factor Binding Specificities: modeling DNA patterns recognized by human regulatory proteins.

🧬 Deciphering human gene regulation is a cornerstone of modern molecular biology and biomedicine. Gene activity is controlled by special regulatory proteins, the transcription factors, which recognize DNA sequence patterns. We invite you to join IBIS in our search for the best method to model binding specificities of yet unexplored human regulatory proteins.

In the challenge, you may use classic methods to represent sequence patterns or any modern approaches
🚀 including decision trees, CNNs, RNNs, LSTMs, and transformers.

💡IBIS allows using arbitrary genome or random sequences to pre-train an artificial neural network or to extract features, and use a few existing datasets, see the IBIS documentation (https://ibis.autosome.org/docs/technical_details).

📊 Yet, the main power and opportunity come with a diverse array of experimental data on 40 human regulatory proteins, many of which remained unexplored until now.

🏆 The best methods will be highlighted in the post-challenge high-impact scientific paper 📝, while the winners 🥇of the Primary track of the Final round will be invited to contribute as co-authors.

🌐 Learn more at https://ibis.autosome.org/
🤗 Our article at HF: https://huggingface.co/blog/nikgr/the-ibis-challenge
👥 Organizers - GRECO-BIT & Codebook consortia: https://ibis.autosome.org/docs/about_us


Sparse Feature Circuits: Discovering and Editing Interpretable Causal
  Graphs in Language Models (2403.19647)

🔍 Today's pick in Interpretability & Analysis of LMs: Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models by @sammarks C. Rager @eircjm @belinkov @davidbau @amueller

This work proposes using features and errors from sparse autoencoders trained to reconstruct LM activations as interpretable units for circuit discovery. The authors then introduce SHIFT, a technique for editing model behavior by ablating interpretable elements from sparse feature circuits. This method is applied alongside unsupervised circuit discovery at scale by means of clustering, showing highly interpretable feature circuits interacting to produce behaviors like predicting sequence increments.

I found the experiment of Section 4 especially convincing and exciting in terms of downstream applications: authors trained a classifier over a biased dataset, and showcased how SHIFT intervention in feature space leads to performances matching those of the same model trained on an unbiased data distribution!

📄 Paper: https://huggingface.co/papers/2403.19647

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9

samusenps nikgr thomwolf osanseviero victor toyxyz


Today we're starting a new initiative: LM Studio Community Models! 🤖

@bartowski, a prolific quantizer (both GGUF and EXL2) will be helping to curate notable new models in LM Studio's Community Models page: https://huggingface.co/lmstudio-community.

Our goal is to ensure the community has access to GGUF files for new & noteworthy models as soon as possible. Keep an eye on that page for updates.

If you're unfamiliar with GGUF, it's the de-facto standard for 'compressed' LLM weights. It is the native format of llama.cpp (https://github.com/ggerganov/llama.cpp, an LLM runtime C/C++ library.) This format is supported in LM Studio.

We will also be sharing new models on the LM Studio Discord: https://discord.gg/aPQfnNkxGC

bartowski victor samusenps Kukedlc InferenceIllusionist bunnycore Severian osanseviero MexIvanov dillfrescott nikgr thomwolf lixin67 julien-c sroecker Midgardsormr AgentHaswing apol British-Rat tcowley


TIL: since Text Generation Inference supports Messages API, which is compatible with the OpenAI Chat Completion API, you can trace calls made to inference endpoints using Langfuse's OpenAI API integration.

A Hugging Face Pro subscription includes access to many models you want to test when developing an app (https://huggingface.co/blog/inference-pro). Using the endpoint and tracing your generations during this development process is an excellent way for GPU-poor people to bootstrap an initial dataset quickly while prototyping.

samusenps lunarflu osanseviero nikgr victor VictorSanh


After giving GPU Programming a hands-on try, I have come to appreciate the level of complexity in AI compute:

- Existing/leading frameworks (CUDA, OpenCL, DSLs, even Triton), still fall at the mercy of low-level compute that requires deeper understanding and experience.
- Ambiguous optimizations methods that will literally drive you mad 🤯
- Triton is cool but not cool enough (high level abstractions that fall back to low level compute issues as you build more specialized kernels)
- As for CUDA, optimization requires considering all major components of the GPU (DRAM, SRAM, ALUs) 🤕
- Models today require stallion written GPU kernels to reduce storage and compute cost.
- GPTQ was a big save 👍🏼

@karpathy  is right expertise in this area is scarce and the reason is quite obvious - uncertainties: we are still struggling to get peak performance from multi-connected GPUs while maintaining precision and reducing cost.

May the Scaling Laws favor us lol.

samusenps lunarflu badayvedat Abecid axwinters


📄 𝗣𝗮𝗽𝗲𝗿 𝗵𝗲𝗿𝗲 👉

📄 𝗣𝗮𝗽𝗲𝗿 𝗵𝗲𝗿𝗲 👉

Mixture-of-Depths: Dynamically allocating compute in transformer-based
  language models (2404.02258)

[𝐍𝐞𝐰 𝐏𝐚𝐩𝐞𝐫] 𝐀𝐥𝐥 𝐭𝐨𝐤𝐞𝐧𝐬 𝐬𝐡𝐨𝐮𝐥𝐝 𝐧𝐨𝐭 𝐫𝐞𝐪𝐮𝐢𝐫𝐞 𝐭𝐡𝐞 𝐬𝐚𝐦𝐞 𝐞𝐟𝐟𝐨𝐫𝐭 𝐭𝐨 𝐜𝐨𝐦𝐩𝐮𝐭𝐞! ⇒ 𝐌𝐢𝐱𝐭𝐮𝐫𝐞 𝐨𝐟 𝐝𝐞𝐩𝐭𝐡𝐬 🫧🐠

Google Researchers were unhappy with the way current decoding generally works: all tokens go through the same layers, thus requiring exactly the same effort to compute.

Whereas in reality, completing the answer to a difficult math problem for instance should be more computationally intense than completing the text of the Declaration of Independence: 𝗻𝗼𝘁 𝗮𝗹𝗹 𝘁𝗼𝗸𝗲𝗻𝘀 𝗮𝗿𝗲 𝗰𝗿𝗲𝗮𝘁𝗲𝗱 𝗲𝗾𝘂𝗮𝗹!

➡ 𝗧𝗵𝗲𝘆 𝗵𝗮𝗱 𝘁𝗵𝗶𝘀 𝗴𝗲𝗻𝗶𝘂𝘀 𝗶𝗱𝗲𝗮: 💡 𝗵𝗮𝘃𝗶𝗻𝗴 𝗮 𝘁𝗼𝗸𝗲𝗻 𝗴𝗼 𝘁𝗵𝗿𝗼𝘂𝗴𝗵 𝗮 𝗯𝗹𝗼𝗰𝗸 𝘀𝗵𝗼𝘂𝗹𝗱 𝗯𝗲 𝗼𝗽𝘁𝗶𝗼𝗻𝗮𝗹. The token can go through the block (thus undergoing expensive self-attention computation) or avoid it through a skip connection.
The routing decision is taken on the block level: each block selects from the total sequence the top-k tokens that will go through it, and the others tokens will skip it. 𝘛𝘩𝘪𝘴 𝘢𝘭𝘭𝘰𝘸𝘴 𝘵𝘰 𝘤𝘩𝘰𝘰𝘴𝘦 𝘵𝘩𝘦 𝘦𝘹𝘢𝘤𝘵 𝙘𝙖𝙥𝙖𝙘𝙞𝙩𝙮 𝘰𝘧 𝘢 𝘣𝘭𝘰𝘤𝘬, 𝘪.𝘦. 𝘵𝘩𝘦 𝘱𝘳𝘰𝘱𝘰𝘳𝘵𝘪𝘰𝘯 𝘰𝘧 𝘵𝘰𝘬𝘦𝘯𝘴 𝘵𝘩𝘢𝘵 𝘨𝘰 𝘵𝘩𝘳𝘰𝘶𝘨𝘩 𝘪𝘵, 𝘸𝘩𝘪𝘤𝘩 𝘥𝘪𝘳𝘦𝘤𝘵𝘭𝘺 𝘪𝘯𝘧𝘭𝘶𝘦𝘯𝘤𝘦𝘴 𝘵𝘩𝘦 𝘤𝘰𝘮𝘱𝘶𝘵𝘢𝘵𝘪𝘰𝘯𝘢𝘭 𝘪𝘯𝘵𝘦𝘯𝘴𝘪𝘵𝘺 𝘰𝘧 𝘵𝘩𝘦 𝘧𝘰𝘳𝘸𝘢𝘳𝘥 𝘱𝘢𝘴𝘴.

This yields Mixture-of-Depths (MoD), with spectacular results.

✨ 𝗥𝗲𝘀𝘂𝗹𝘁𝘀:
🎚 𝗖𝗮𝗽𝗮𝗰𝗶𝘁𝘆 𝗰𝗮𝗻 𝗯𝗲 𝘁𝘂𝗻𝗲𝗱 𝗮𝗹𝗹 𝘁𝗵𝗲 𝘄𝗮𝘆 𝗱𝗼𝘄𝗻 𝘁𝗼 𝟭𝟮.𝟱% for every second block: thus 87.5% of tokens just skip the block!
🚀 For the same training time and performance, >𝟲𝟬% 𝗶𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝘀𝗽𝗲𝗲𝗱!
🤝 𝗖𝗮𝗻 𝗯𝗲 𝗰𝗼𝗺𝗯𝗶𝗻𝗲𝗱 𝘄𝗶𝘁𝗵 𝗠𝗶𝘅𝘁𝘂𝗿𝗲-𝗼𝗳-𝗘𝘅𝗽𝗲𝗿𝘁𝘀 for further improvements.

📄 𝗣𝗮𝗽𝗲𝗿 𝗵𝗲𝗿𝗲 👉 https://huggingface.co/papers/2404.02258
📚 I added it to my paper collection 👉 https://huggingface.co/collections/m-ric/spinning-up-in-llms-659e698f9dd5a71bd3f579a7

clem samusenps ABX-AI mmhamdy nikgr minjejeon qinluo MultiTrickFox mammour danielus Joseph717171 ivanfioravanti


Sailor: Open Language Models for South-East Asia (2404.03608)

⚓ Sailor: A New Multilingual Open LLM for South-East Asia 🌏

Last month we have released a new family of multilingual language models called **Sailor**, ranging from 0.5B to 7B parameters, continually pre-trained from the Qwen1.5 models. Based on our extensive benchmarking, the Sailor models demonstrate exceptional performance on South-East Asian languages, taking us one step closer to multilingual LLMs that can serve the diverse needs of the region and beyond.

Today, we're more than excited to share the key technical details behind the Sailor models! 💪

**Key highlights**:
🔍 Data curation: Merging short examples, document-level code-switching, aggressive data cleaning and deduplication.
🤖 Tokenization Robustness: We find that BPE dropout is really effective to deal with prompt variations.
🔍 Optimizing Data Mixture: We propose a new approach to automatically balance capabilities across different languages!
🌟 Recipe in Continual Pre-training: We discover a powerful metric that can help predict how well the Sailor models will perform on the original domain (e.g., English) after continual pre-training.

We are thrilled to share these technical details with the community and invite you to explore the Sailor models. We hope Sailor models take us one step closer to multilingual LLMs in the world! 🌍✨

To learn more, please access our research paper or reach out to our team.
🔗 Paper: https://huggingface.co/papers/2404.03608
🧩 Model: https://huggingface.co/collections/sail/sailor-language-models-65e19a749f978976f1959825
💻 Code: https://github.com/sail-sg/sailor-llm

dreamerdeo Jiiiia teddythinh chaoscodes DmitryRyumin taufiqdp SivilTaram osanseviero clem victor lunarflu Kernel nikgr gabrielchua DongfuJiang damerajee yjernite

dreamerdeo Jiiiia teddythinh aboutyourresearch chaoscodes monsoon-nlp osanseviero clem SivilTaram samusenps lunarflu damerajee yjernite


Mixture-of-Depths

Dynamically allocating compute in transformer-based language models

https://huggingface.co/papers/2404.02258

Transformer-based language models spread FLOPs uniformly across input sequences. In this work we demonstrate that transformers can instead learn to dynamically allocate FLOPs (or compute) to specific positions in a sequence, optimising the allocation along the sequence for different layers across the model depth. Our method enforces a total compute budget by capping the number of tokens (k) that can participate in the self-attention and MLP computations at a given layer. The tokens to be processed are determined by the network using a top-k routing mechanism. Since k is defined a priori, this simple procedure uses a static computation graph with known tensor sizes, unlike other conditional computation techniques. Nevertheless, since the identities of the k tokens are fluid, this method can expend FLOPs non-uniformly across the time and model depth dimensions. Thus, compute expenditure is entirely predictable in sum total, but dynamic and context-sensitive at the token-level. Not only do models trained in this way learn to dynamically allocate compute, they do so efficiently. These models match baseline performance for equivalent FLOPS and wall-clock times to train, but require a fraction of the FLOPs per forward pass, and can be upwards of 50\% faster to step during post-training sampling.

samusenps taufiqdp clem mick-bitovi iancos csouza nikgr rupeshs sugatoray dillfrescott ceshine


Introducing https://huggingface.co/datasets/gretelai/synthetic_text_to_sql by https://huggingface.co/gretelai

It stands as the largest and most diverse synthetic Text-to-SQL dataset available to-date.

The dataset includes:

- 105,851 records partitioned into 100,000 train and 5,851 test records
~23M total tokens, including ~12M SQL tokens
- Coverage across 100 distinct domains/verticals
- Comprehensive array of SQL tasks: data definition, retrieval, manipulation, analytics & reporting
- Wide range of SQL complexity levels, including subqueries, single joins, multiple joins, aggregations, window functions, set operations
- Database context, including table and view create statements
- Natural language explanations of what the SQL query is doing
- Contextual tags to optimize model training

Blogpost: https://gretel.ai/blog/synthetic-text-to-sql-dataset
Dataset: https://huggingface.co/datasets/gretelai/synthetic_text_to_sql

ichigoberry mvaloatto Svngoku nikgr thomwolf lhoestq brhnsbn


GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from
  a Single Image (2404.02152)

🚀🎭🌟 New Research Alert - CVPR 2024 (Avatars Collection)! 🌟 🎭🚀
📄 Title: GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image 🔝

📝 Description: GeneAvatar is a generic approach for editing 3D head avatars based on a single 2D image, applicable to different volumetric representations. The novel expression-aware generative modification model delivers high quality and consistent editing results across multiple viewpoints and emotions.

👥 Authors: Chong Bao et al.

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2404.02152

🌐 Github Page: https://zju3dv.github.io/geneavatar/
📁 Repository: https://github.com/zju3dv/GeneAvatar

📺 Video: https://www.youtube.com/watch?v=4zfbfPivtVU

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #GeneAvatar #HeadAvatar #3DHeadAvatarEditing #VolumetricHeadAvatar #SingleImageEditing #ExpressionAwareModification #CVPR2024 #DeepLearning #Innovation

dashfunnydashdash samusenps Limbicnation MichaelKwak osanseviero clem introvoyz041 Benson Alkohole

DmitryRyumin samusenps osanseviero Kukedlc clem


Introducing the Gradio API Recorder 🪄

Every Gradio app now includes an API recorder that lets you reconstruct your interaction in a Gradio app as code using the Python or JS clients! Our goal is to make Gradio the easiest way to build ML APIs, not just UIs 🔥

Bagus victor merve gabrielchua samusenps radames WhatIFTheMind osanseviero ademait brianjking RocktProductions

DmitryRyumin merve nikgr samusenps radames jeffboudier osanseviero

DmitryRyumin victor merve radames osanseviero


InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image
  Generation (2404.02733)

Greeting!

We are happy to introduce our InstantStyle, which is a framework that employs straightforward yet potent techniques for achieving effective disentanglement of style and content from reference images.

Code: https://github.com/InstantStyle/InstantStyle
Paper: https://huggingface.co/papers/2404.02733
Project Page: https://instantstyle.github.io/

DmitryRyumin samusenps victor merve tonyassi namelysweet thomwolf badayvedat dillfrescott penouc


ARABIC CHINESE FRENCH GERMAN RUSSIAN SPANISH TURKISH

mLLM - first release:
orca_dpo_pairs by Intel (translated into 7 languages)

ARABIC CHINESE FRENCH GERMAN RUSSIAN SPANISH TURKISH

Upcoming:
- more datasets
- cleaning steps
- a blogpost
- stay updated at https://hf.co/multilingual

https://huggingface.co/datasets/multilingual/orca_dpo_pairs

samusenps monsoon-nlp seyf1elislam aari1995 osanseviero victor merve nikgr Kukedlc


🆕 Releasing a new series of 8 zeroshot classifiers: better performance, fully commercially useable thanks to synthetic data, up to 8192 tokens, run on any hardware.

Summary:
🤖 The zeroshot-v2.0-c series replaces commercially restrictive training data with synthetic data generated with mistralai/Mixtral-8x7B-Instruct-v0.1 (Apache 2.0). All models are released under the MIT license.
🦾 The best model performs 17%-points better across 28 tasks vs. facebook/bart-large-mnli (the most downloaded commercially-friendly baseline).
🌏 The series includes a multilingual variant fine-tuned from BAAI/bge-m3 for zeroshot classification in 100+ languages and with a context window of 8192 tokens
🪶 The models are 0.2 - 0.6 B parameters small, so they run on any hardware. The base-size models are +2x faster than bart-large-mnli while performing significantly better.
🤏 The models are not generative LLMs, they are efficient encoder-only models specialized in zeroshot classification through the universal NLI task.
🤑 For users where commercially restrictive training data is not an issue, I've also trained variants with even more human data for improved performance.

Next steps:
✍ I'll publish a blog post with more details soon
🔮 There are several improvements I'm planning for v2.1. Especially the multilingual model has room for improvement.

All models are available for download in this Hugging Face collection: https://huggingface.co/collections/MoritzLaurer/zeroshot-classifiers-6548b4ff407bb19ff5c3ad6f

These models are an extension of the approach explained in this paper, but with additional synthetic data: https://arxiv.org/abs/2312.17543

victor tomaarsen samusenps clem vladbogo jeffboudier mmhamdy osanseviero peter2000 merve peycke


👋 👋 👋  In a recent AI Time Journal Interview, I was asked:  How has HackerNoon leveraged AI to enhance its platform? and what lessons have you learned about AI’s role in spam creation and prevention?

https://HackerNoon.com  uses AI and machine learning to improve grammar, translate stories, generate featured images, suggest possible headlines, (and note, I would love to hear what else you think we should use AI for!) . Maybe Clippy was ahead of its time? Writers have and will continue to have smart AI assistants. AI can not and should not replace the writer, but it should and can assist the writer and editor throughout the publishing process. We are documenting our approach to editorial at https://EditingProtocol.com. With the rise of LLMs, we’ve seen an influx of AI story spam submissions. Other blogging platforms are hosting so many of these types of machine-generated stories right now. We are actively taking steps not to become a wasteland of AI’s musings. Because every story is reviewed before publication (with about half being rejected), I think our average story quality is higher than other blogging platforms. Every story submission automatically enters into plagiarism and AI writing detection. We highlight what sections are likely, possibly, and unlikely to be written by AI. It’s all about confidence levels. Ultimately, human editors make the final call in interpreting the machines’ reports.


Advancing LLM Reasoning Generalists with Preference Trees (2404.02078)

Advancing LLM Reasoning Generalists with Preference Trees

https://huggingface.co/papers/2404.02078

We introduce Eurus, a suite of large language models (LLMs) optimized for reasoning. Finetuned from Mistral-7B and CodeLlama-70B, Eurus models achieve state-of-the-art results among open-source models on a diverse set of benchmarks covering mathematics, code generation, and logical reasoning problems. Notably, Eurus-70B beats GPT-3.5 Turbo in reasoning through a comprehensive benchmarking across 12 tests covering five tasks, and achieves a 33.3% pass@1 accuracy on LeetCode and 32.6% on TheoremQA, two challenging benchmarks, substantially outperforming existing open-source models by margins more than 13.3%. The strong performance of Eurus can be primarily attributed to UltraInteract, our newly-curated large-scale, high-quality alignment dataset specifically designed for complex reasoning tasks. UltraInteract can be used in both supervised fine-tuning and preference learning. For each instruction, it includes a preference tree consisting of (1) reasoning chains with diverse planning strategies in a unified format, (2) multi-turn interaction trajectories with the environment and the critique, and (3) pairwise data to facilitate preference learning. UltraInteract allows us to conduct an in-depth exploration of preference learning for reasoning tasks. Our investigation reveals that some well-established preference learning algorithms may be less suitable for reasoning tasks compared to their effectiveness in general conversations. Inspired by this, we derive a novel reward modeling objective which, together with UltraInteract, leads to a strong reward model.

samusenps clem vladbogo merve nikgr Bruno


Introducing VLRM: Vision-Language Models act as Reward Models for Image Captioning
Paper:
https://arxiv.org/abs/2404.01911
Model weights and training logs:
https://huggingface.co/sashakunitsyn/vlrm-blip2-opt-2.7b

We propose VLRM, a method for fine-tuning an existing image captioning model using reinforcement learning and vision-language models as reward models.
The method manages to significantly improve the generation quality without human-labeled data and is applicable to any image captioning model.
Our model reaches impressive 0.90 R@1 CLIP Recall score on MS-COCO Carpathy Test Split.

samusenps nikgr victor clem osanseviero merve Kukedlc AlkaouDembele xziayro


Efficient 3D Implicit Head Avatar with Mesh-anchored Hash Table
  Blendshapes (2404.01543)

🚀🎭🌟 New Research Alert - CVPR 2024 (Avatars Collection)! 🌟 🎭🚀
📄 Title: MonoAvatar++: Efficient 3D Implicit Head Avatar with Mesh-anchored Hash Table Blendshapes 🔝

📝 Description: MonoAvatar++ is a real-time neural implicit 3D head avatar model with high quality and fine-grained control over facial expressions. It uses local hash table blendshapes attached to a parametric facial model for efficient rendering, achieving SOTA results even for challenging expressions.

👥 Authors: Ziqian Bai, Feitong Tan, Sean Fanello, Rohit Pandey, Mingsong Dou, Shichen Liu, Ping Tan, Yinda Zhang

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2404.01543

🌐 Github Page: https://augmentedperception.github.io/monoavatar-plus

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #MonoAvatar++ #HeadAvatar #3DModeling #AvatarGeneration #NeuralImplicitAvatar #EfficientRendering #CVPR2024 #DeepLearning #Innovation

DmitryRyumin samusenps nikgr victor clem merve


ChemLLM version 1.5 has been open sourced. It has also released a version that has undergone human feedback reinforcement learning. In particular, the front-end and back-end codes have been open sourced simultaneously to facilitate deployment. Everyone is welcome to pay attention.
The evaluation set and part of the training set will also be contributed to the community in days, so stay tuned ✧٩(ˊωˋ*)و✧

ChemLLM-1.5
https://huggingface.co/AI4Chem/ChemLLM-7B-Chat-1.5-DPO

ChemLLM-1.5-dpo https://huggingface.co/AI4Chem/ChemLLM-7B-Chat-1.5-SFT

Front-end https://github.com/AI4Chem/NextChat
RAG application example https://github.com/AI4Chem/NextChat/blob/main/chatpdf.py
Backend https://github.com/AI4Chem/api_backend

Front-end Demo https://chemllm.org/
RAG application example Demo https://chatpdf.chemllm.org/
Backend Demo https://api.chemllm.org/

qq8933 osanseviero ajibawa-2023 samusenps nikgr clem merve mmhamdy dillfrescott


Following up on @vikhyatk's Moondream2 update and @santiagomed's implementation on Candle, I quickly put togheter the WASM module so that you could try running the ~1.5GB quantized model in the browser. Perhaps the next step is to rewrite it using https://github.com/huggingface/ratchet and run it even faster with WebGPU, @FL33TW00D-HF.

https://huggingface.co/spaces/radames/Candle-Moondream-2

ps: I have a collection of all Candle WASM demos here https://huggingface.co/collections/radames/candle-wasm-examples-650898dee13ff96230ce3e1f

vikhyatk FL33TW00D-HF samusenps nikgr clem merve Felladrin osanseviero

vikhyatk FL33TW00D-HF clem merve kylen osanseviero


Gradio 4.25.0 is out with some nice improvements and bug fixes!

🧹 Automatic deletion of gr.State variables stored in the server. Never run out of RAM again. Also adds an unload event you can run when a user closes their browser tab.

😴 Lazy example caching. You can set cache_examples="lazy" to cache examples when they're first requested as opposed to before the server launches. This can cut down the server's start-up time drastically.

🔊 Fixes a bug with streaming audio outputs

🤖 Improvements to gr.ChatInterface like pasting images directly from the clipboard.

See the rest of the changelog here: https://www.gradio.app/changelog#4-25-0

samusenps ysharma not-lain ajibawa-2023 radames DmitryRyumin nikgr clem clefourrier akhaliq on1onmangoes


Released a new version of https://huggingface.co/vikhyatk/moondream2 today! Primarily focused on improving OCR and captioning (e.g. "Describe this image", "Describe this image in one sentence"), but also seeing general improvement across all benchmarks.

not-lain Corianas ajibawa-2023 radames DmitryRyumin nikgr clem xingren23 dillfrescott aaryanverma linz Jorgegarciairazabal Vlansu

not-lain clem sooperDime Sylvestre dillfrescott omaryshchenko Jorgegarciairazabal


These 15 open models are available for serverless inference on Cloudflare Workers AI, powered by GPUs distributed in 150 datacenters globally - 👏 @rita3ko @mchenco @jtkipp @nkothariCF @philschmid

https://huggingface.co/collections/Cloudflare/hf-curated-models-available-on-workers-ai-66036e7ad5064318b3e45db6

rwightman nkothariCF samusenps ajibawa-2023 philschmid radames victor nikgr kramp clem Tonic

samusenps philschmid radames victor clem Tonic


Anthropic introduces "Many-shot Jailbreaking" (MSJ), a new attack on large language models! MSJ exploits long context windows to override safety constraints.

Key Points:
* Prompts LLMs with hundreds of examples of harmful behavior formatted as a dialogue
* Generates malicious examples using an uninhibited "helpful-only" model
* Effective at jailbreaking models like Claude 2.0, GPT-3.5, GPT-4
* Standard alignment techniques provide limited protection against long context attacks

Paper: https://www.anthropic.com/research/many-shot-jailbreaking
More details in my blog: https://huggingface.co/blog/vladbogo/many-shot-jailbreaking

Congrats to the authors for their work!


𝟐𝟎𝟐𝟒, 𝐭𝐡𝐞 𝐲𝐞𝐚𝐫 𝐨𝐟 𝐚𝐠𝐞𝐧𝐭 𝐰𝐨𝐫𝐤𝐟𝐥𝐨𝐰𝐬 🔧🦾🤖

𝟐𝟎𝟐𝟒, 𝐭𝐡𝐞 𝐲𝐞𝐚𝐫 𝐨𝐟 𝐚𝐠𝐞𝐧𝐭 𝐰𝐨𝐫𝐤𝐟𝐥𝐨𝐰𝐬 🔧🦾🤖

𝗪𝗵𝘆 𝘂𝘀𝗲 𝗮𝗴𝗲𝗻𝘁 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀?

𝗪𝗵𝘆 𝘂𝘀𝗲 𝗮𝗴𝗲𝗻𝘁 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀?

𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗱𝗲𝘀𝗶𝗴𝗻 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀

𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗱𝗲𝘀𝗶𝗴𝗻 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀

𝗖𝗹𝗼𝘀𝗶𝗻𝗴 𝘁𝗵𝗼𝘂𝗴𝗵𝘁𝘀

𝗖𝗹𝗼𝘀𝗶𝗻𝗴 𝘁𝗵𝗼𝘂𝗴𝗵𝘁𝘀

𝟐𝟎𝟐𝟒, 𝐭𝐡𝐞 𝐲𝐞𝐚𝐫 𝐨𝐟 𝐚𝐠𝐞𝐧𝐭 𝐰𝐨𝐫𝐤𝐟𝐥𝐨𝐰𝐬 🔧🦾🤖

I've just watched Andrew Ng's talk at Sequoia last week.
If you're interested in Agents, you should really watch it!

𝗪𝗵𝘆 𝘂𝘀𝗲 𝗮𝗴𝗲𝗻𝘁 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀?
The current LLM task solving workflow is not very intuitive:
We ask it “write an essay all in one shot, without ever using backspace.”

Why not allow the LLM a more similar process to what we would do?
- “Write an essay outline”
- “Do you need wen research?”
- “Write a first draft”
- “Consider improvements”
…

This is called an Agentic workflow. Existing ones bring a huge performance boost. With HumanEval: GPT-4 zero-shot gets 67% score, agentic with either one of tool use or reflection goes over 90%, and the combination of the two scores even higher!

𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗱𝗲𝘀𝗶𝗴𝗻 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀
On the following two points, the tech is robust:

⚙ 𝗥𝗲𝗳𝗹𝗲𝘅𝗶𝗼𝗻: For instance: add a critic step after the writing step
🛠 𝗧𝗼𝗼𝗹 𝘂𝘀𝗲: extends the capabilities of the LLM by allowing it to call tools, like search or calculator

The next two will be needed to go further, but the tech for them is more emerging and not reliable yet:
🗺 𝗣𝗹𝗮𝗻𝗻𝗶𝗻𝗴 forward to decompose task into subtasks. This allows great behaviours like an AI Agent re-routing after a failure
🐝 𝗠𝘂𝗹𝘁𝗶-𝗮𝗴𝗲𝗻𝘁 𝗰𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝗼𝗻: Program a flock of agents with tasks.
Improving the two above points will unlock huge performance boosts!

Andrew NG says Research agents are already part of his workflow!

𝗖𝗹𝗼𝘀𝗶𝗻𝗴 𝘁𝗵𝗼𝘂𝗴𝗵𝘁𝘀
Andrew speculates that through agentic workflows, maybe generating many tokens fast from a small LLM will give better results than slower throughput from a powerful LLM like GPT-5.

🎬 Watch the talk here 👉 https://www.youtube.com/watch?v=sal78ACtGTc
📚 I've added his recommended reads to https://huggingface.co/collections/m-ric/agents-65ba776fbd9e29f771c07d4e

osanseviero clem neelam91721 samusenps nikgr JoPmt Moibe abidlabs poGlingus


Aurora-M: The First Open Source Multilingual Language Model Red-teamed
  according to the U.S. Executive Order (2404.00399)

Aurora-M

The First Open Source Multilingual Language Model Red-teamed according to the U.S. Executive Order

https://huggingface.co/papers/2404.00399

Pretrained language models underpin several AI applications, but their high computational cost for training limits accessibility. Initiatives such as BLOOM and StarCoder aim to democratize access to pretrained models for collaborative community development. However, such existing models face challenges: limited multilingual capabilities, continual pretraining causing catastrophic forgetting, whereas pretraining from scratch is computationally expensive, and compliance with AI safety and development laws. This paper presents Aurora-M, a 15B parameter multilingual open-source model trained on English, Finnish, Hindi, Japanese, Vietnamese, and code. Continually pretrained from StarCoderPlus on 435 billion additional tokens, Aurora-M surpasses 2 trillion tokens in total training token count. It is the first open-source multilingual model fine-tuned on human-reviewed safety instructions, thus aligning its development not only with conventional red-teaming considerations, but also with the specific concerns articulated in the Biden-Harris Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. Aurora-M is rigorously evaluated across various tasks and languages, demonstrating robustness against catastrophic forgetting and outperforming alternatives in multilingual settings, particularly in safety evaluations.

osanseviero CulturedMan DmitryRyumin samusenps clem Catliba mayank-mishra sugatoray


How would you benchmark performance estimation algorithms vs data drift signals?

I'm working on a benchmarking analysis, and I'm currently doing the following:

-  Get univariate and multivariate drift signals and measure their correlation with realized performance.
- Use drift signals as features of a regression model to predict the model's performance.
- Use drift signals as features of a classification model to predict a performance drop.
- Compare all the above experiments with results from Performance Estimation algorithms.

Any other ideas?


🥇Open CoT Leaderboard

We're delighted to announce the [Open CoT Leaderboard](https://huggingface.co/spaces/logikon/open_cot_leaderboard) on 🤗 Spaces.

Unlike other LLM performance leaderboards, the Open CoT Leaderboard is not tracking absolute benchmark accuracies, but relative **accuracy gains** due to **chain-of-thought**.

Eval datasets that underpin the leaderboard are hosted [here](https://huggingface.co/cot-leaderboard).

Feedback and suggestions more than welcome.

@clefourrier

clefourrier lunarflu osanseviero samusenps giux78 davidhought0n reuank victor nikgr yakazimir clem


We would like to announce our Aurora-M multilingual models which is based on Starcoderplus.
Twitter: https://twitter.com/ontocord/status/1772778544051155029
LinkedIn: https://www.linkedin.com/feed/update/urn:li:activity:7178521998845759488/
Blog post: https://huggingface.co/blog/mayank-mishra/aurora
Arxiv: https://huggingface.co/papers/2404.00399

Current LLMs are very susceptible to generating toxic, harmful and even dangerous content. They can also generate outputs with gender or racial biases. The Biden-Harris Executive Order https://www.federalregister.gov/documents/2023/11/01/2023-24283/safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence) sets forth guidelines on what is considered a safe AI system.
Following up on these guidelines, we present the world's first open source Biden-Harris Executive Order Red teamed Multilingual Language Model: Aurora-M. Inspired by BigScience, the model is trained on 5 languages: English, Hindi, Japanese, Vietnamese and Finnish.

* Red teamed model: https://huggingface.co/aurora-m/aurora-m-biden-harris-redteamed(safety tuned according to the order mentioned above)
* Base model: https://huggingface.co/aurora-m/aurora-m-base (not safety tuned)
* Instruct model: https://huggingface.co/aurora-m/aurora-m-instruct (not safety tuned)

@mayank-mishra @cabbage972 @sted97 @Xa9aX @Taishi-N324 @Muennighoff @vumichien @prateeky2806 @felfri @spyysalo and many many others!

samusenps Xa9aX lunarflu osanseviero mayank-mishra vumichien kenhktsui nikgr sted97 muhtasham Taishi-N324 Gokumase

samusenps Xa9aX lunarflu osanseviero mayank-mishra vumichien kenhktsui sted97 Taishi-N324


⚡ AutoQuant

AutoQuant is the evolution of my previous AutoGGUF notebook (https://colab.research.google.com/drive/1P646NEg33BZy4BfLDNpTz0V0lwIU3CHu). It allows you to quantize your models in five different formats:

- GGUF: perfect for inference on CPUs (and LM Studio)
- GPTQ/EXL2: fast inference on GPUs
- AWQ: super fast inference on GPUs with vLLM (https://github.com/vllm-project/vllm)
- HQQ: extreme quantization with decent 2-bit and 3-bit models

Once the model is converted, it automatically uploads it on the Hugging Face Hub. To quantize a 7B model, GGUF only needs a T4 GPU, while the other methods require an A100 GPU.

Here's an example of a model I quantized using HQQ and AutoQuant: https://huggingface.co/mlabonne/AlphaMonarch-7B-2bit-HQQ

I hope you'll enjoy it and quantize lots of models! :)

💻 AutoQuant: https://colab.research.google.com/drive/1b6nqC7UZVt8bx4MksX7s656GXPM-eWw4

marcsun13 mishig Outrun32 johko kgourgou DataSoul samusenps lunarflu osanseviero julien-c OxxoCodes musfiqdehan seyf1elislam nasiruddin15 Heng666 nikgr sbarman25 matlok reach-vb AlekseiPravdin maaarcooo aspenita victor sbrandeis Kukedlc

eldogbbhed kgourgou louisbrulenaudet reach-vb multitude0099 DavidAU AlekseiPravdin ManniX-ITA nguyenthanhdo Kukedlc


Symbol-LLM: Towards Foundational Symbol-centric Interface For Large
  Language Models (2311.09278)

Check out our work Symbol-LLM ! We have open-sourced both 7B / 13B model weights, as well as part of the symbolic collections ! Try it !

Paper link: https://huggingface.co/papers/2311.09278

Model weights: https://huggingface.co/Symbol-LLM/Symbol-LLM-7B-Instruct

Symbol-LLM xufangzhi FeYuan vladbogo lunarflu osanseviero nikgr

xufangzhi FeYuan QiushiSun lunarflu osanseviero


Introducing Indic Chat!

Try out best opensource Indic LLMs now on https://www.indic.chat/

Models available:
• Telugu-LLM-Labs/Indic-gemma-7b-finetuned-sft-Navarasa-2.0
• GenVRadmin/AryaBhatta-GemmaOrca
• BhabhaAI/Gajendra-v0.1
• ai4bharat/Airavata

Additionally:

1. We open up our discord for everyone to collaborate & accelerate Indic LLMs: https://bhabha.ai/discord

2. We release ~600K rows filtered & Hindi translated version of OpenHermes-2.5 instruction dataset: https://huggingface.co/datasets/BhabhaAI/openhermes-2.5-hindi

Also, thanks to our compute sponsor - Telugu LLM Labs & Bhabha AI in helping us serve models for Indic Chat.

If you’d like to be a sponsor too, checkout
https://www.indic.chat/sponsor

monsoon-nlp lunarflu osanseviero alielfilali01 ajibawa-2023 nikgr ashimksaha Saugatkafley

samusenps lunarflu osanseviero alielfilali01


On evaluating fine tuned 7B Italian open source LLMs I have collected many data points and I created a super simple explorative analyses. My hypothesis  based on data are:

 - mmlu is hard to improve when fine tuning a base model on a different language
- fine tuning also on single GPUs can improve by 5% to 10% the base model on common tasks but a lot more on specific cases with the right training time and data
- fine tuning can specialize well but at cost of loosing some foundational  knowledge.

Here the data https://docs.google.com/spreadsheets/d/1MBcxy1loK8eIycZG4DN84Q2ejZ0jSjxUBgoShHDR6IY/edit?usp=sharing
Here the colab https://colab.research.google.com/drive/1ra4_skG5QYWSYOzvagOoIoj4bibQD8Gw?usp=sharing
Here an article with some considerations https://medium.com/@giuxale/an-analyses-on-italian-llms-models-evaluations-51bffe1d44d1


Gecko: Versatile Text Embeddings Distilled from Large Language Models (2403.20327)

Google DeepMind introduces Gecko a new text embedding! Gecko uses a two-step process that leverages synthetic data generation and reranking.

Keypoints:
* Uses an LLM to generate diverse synthetic queries and tasks from web passages
* Refines the data by retrieving candidate passages and relabeling positives/negatives using the same LLM
* Achieves very good results on the Massive Text Embedding Benchmark, where compact 256D Gecko outperforms 768D models.
* 768D Gecko achieves state-of-the-art performance competing with models a lot larger larger.

Paper: https://huggingface.co/papers/2403.20327
More details in my blog:  https://huggingface.co/blog/vladbogo/gecko

Congrats to the authors for their work!

samusenps clem monsoon-nlp beekay24 mohammedbriman lunarflu nikgr tatv047 super-cinnamon


Jamba: A Hybrid Transformer-Mamba Language Model (2403.19887)

Jamba

A Hybrid Transformer-Mamba Language Model

https://huggingface.co/papers/2403.19887

We present Jamba, a new base large language model based on a novel hybrid Transformer-Mamba mixture-of-experts (MoE) architecture. Specifically, Jamba interleaves blocks of Transformer and Mamba layers, enjoying the benefits of both model families. MoE is added in some of these layers to increase model capacity while keeping active parameter usage manageable. This flexible architecture allows resource- and objective-specific configurations. In the particular configuration we have implemented, we end up with a powerful model that fits in a single 80GB GPU. Built at large scale, Jamba provides high throughput and small memory footprint compared to vanilla Transformers, and at the same time state-of-the-art performance on standard language model benchmarks and long-context evaluations. Remarkably, the model presents strong results for up to 256K tokens context length. We study various architectural decisions, such as how to combine Transformer and Mamba layers, and how to mix experts, and show that some of them are crucial in large scale modeling. We also describe several interesting properties of these architectures which the training and evaluation of Jamba have revealed, and plan to release checkpoints from various ablation runs, to encourage further exploration of this novel architecture. We make the weights of our implementation of Jamba publicly available under a permissive license.

samusenps osanseviero Benyucong clem beekay24 mohammedbriman vladbogo adhi01 lunarflu nikgr kramp ABX-AI


🎯🖼🌟 New Research Alert - ICLR 2024! 🌟 🖼🎯
📄 Title: Adversarial AutoMixup 🖼

📝 Description: Adversarial AutoMixup is an approach to image classification augmentation. By alternately optimizing a classifier and a mixed-sample generator, it attempts to generate challenging samples and improve the robustness of the classifier against overfitting.

👥 Authors: Huafeng Qin et al.

📅 Conference: ICLR, May 7-11, 2024 | Vienna, Austria 🇦🇹

🔗 Paper: https://huggingface.co/papers/2312.11954

📁 Repository: https://github.com/JinXins/Adversarial-AutoMixup

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🔍 Keywords: #AutoMixup #ImageClassification #ImageAugmentation #AdversarialLearning #ICLR2024 #DeepLearning #Innovation

samusenps osanseviero clem beekay24 lunarflu atrxial MiSTe-R nikgr PantheonAI


Little know gem: the Open-source Cookbook

A collection of notebooks for building practical AI applications using open-source tools and models: https://lnkd.in/e6m6Jmwu

Doc: https://lnkd.in/e3FE6TUq

Currently contains 16 notebooks in English (and some in Chinese):
	1.	Using LLM-as-a-judge 🧑⚖ for an automated and versatile evaluation
	2.	Create a legal preference dataset
	3.	Suggestions for Data Annotation with SetFit in Zero-shot Text Classification
	4.	Implementing semantic cache to improve a RAG system
	5.	Building A RAG Ebook “Librarian” Using LlamaIndex
	6.	Stable Diffusion Interpolation
	7.	Building A RAG System with Gemma, MongoDB and Open Source Models
	8.	Prompt Tuning with PEFT Library
	9.	Migrating from OpenAI to Open LLMs Using TGI’s Messages API
	10.	Automatic Embeddings with TEI through Inference Endpoints
	11.	Simple RAG for GitHub issues using Hugging Face Zephyr and LangChain
	12.	Embedding multimodal data for similarity search using 🤗 transformers, 🤗 datasets and FAISS
	13.	Fine-tuning a Code LLM on Custom Code on a single GPU
	14.	RAG Evaluation Using Synthetic data and LLM-As-A-Judge
	15.	Advanced RAG on HuggingFace documentation using LangChain
	16.	Detecting Issues in a Text Dataset with Cleanlab

giux78 samusenps osanseviero clem beekay24 ajibawa-2023 vladbogo peaceAsh lunarflu nikgr vikas

samusenps giux78 osanseviero clem lunarflu anandamide Srjnnnn OmbelineM


The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits (2402.17764)

SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation (2309.17448)

Diaries of Open Source. Part 13 🤗

🤏Two different bitnet 1.5 open-source replications
Original paper: https://hf.co/papers/2402.17764
1bitllm experiment: https://hf.co/blog/joey00072/experiments-with-bitnet-1-5
NousResearch experiment https://hf.co/NousResearch/OLMo-Bitnet-1B

 🥳Tiny and large multimodal models great for embeddings
GitHub: https://github.com/unum-cloud/uform
Encoders: https://hf.co/collections/unum-cloud/multimodal-encoders-660553903617c5297eb16838
ONNX weights: https://hf.co/collections/unum-cloud/uform-vl-english-large-onnx-66055a57c182d846f3bc1949

📜 SMPLer-X: Expressive Human Pose and Shape Estimation
Project website: https://caizhongang.com/projects/SMPLer-X/
Demo: https://huggingface.co/spaces/caizhongang/SMPLer-X
Paper: https://hf.co/papers/2309.17448

🧙GeoWizard: 3D Geometry Estimation
Project website: https://fuxiao0719.github.io/projects/geowizard/
Demo: https://hf.co/spaces/lemonaddie/geowizard

Misc models and datasets
- Dolphin-2.8-mistral-7b-v0.2 https://hf.co/cognitivecomputations/dolphin-2.8-mistral-7b-v02
- Hermes-2-Pro-11B,  a self-frankenmerge 11B variant https://hf.co/mattshumer/Hermes-2-Pro-11B
- Large conversational dataset based on Usenet data in the Italian language https://hf.co/datasets/mii-community/UsenetArchiveIT-conversations

ruggsea mrinaldi DmitryRyumin samusenps giux78 clem vladbogo lunarflu nikgr abidlabs


This is the closest I’ve seen of a scalable AI/LLM Operating System - it has all the major ingredients of a feasible AI OS 1 architecture:

- Extends classical OS functionalities with an LLM Kernel.
- Multi agent-centric approach.
- Optimized resource allocation system that allows for LLM-based tasks and Classical OS tasks to coexist.
- An Agent Scheduler that can perform classical os operations (FIFO, RR).
- A Context Manager to improve alignment.
- Lazy Memory Manager for agents (ensures data is stored and accessible only while the agent is active)
- An Enhanced security module for the AI-driven environment.

It does hit all checkpoints, doesn’t it? An upscale version of @karpathy’s.

Code: https://github.com/agiresearch/AIOS

ajibawa-2023 osanseviero phuong-d-h-nguyen Dlbk rsamantha KvrParaskevi AIAdventurist JoshuaGu NeuralNovel thomwolf kazeMace disham993 kubo-vn abhinit21 clem jhondavid DataSoul alielfilali01 lunarflu nikgr Skier8402

samusenps tatoy JoseRFJunior thomwolf Nakini disham993 lunarflu

chuangxinlezhi disham993 KIKININA lunarflu


🚀🚀 Exciting times for the document AI community!

We're thrilled to announce the release of some of the largest OCR datasets available to the public.
🔥 With over 26 million pages , 18 billion text tokens, and 6TB of data, these resources are a significant leap forward for document AI research.

Here's how to access these datasets quickly:

```
from datasets import load_dataset

pdfa_dataset = load_dataset('pixparse/pdfa-eng-wds', streaming=True)
IDL_dataset = load_dataset('pixparse/idl-wds', streaming=True)
```
This enables you to stream them directly, integrating seamlessly with your projects using the Hugging Face datasets library. On the hub, you can find them here:

https://huggingface.co/datasets/pixparse/pdfa-eng-wds
https://huggingface.co/datasets/pixparse/idl-wds

For lean data loading, the new [chug](https://github.com/huggingface/chug) library offers a solution with pdf decoding:

```
import chug

task_cfg = chug.DataTaskDocReadCfg(
    page_sampling='all',
)
data_cfg = chug.DataCfg(
    source='pixparse/pdfa-eng-wds',
    split='train',
    batch_size=None,
    format='hfids',
    num_workers=0,
)
data_loader = chug.create_loader(
    data_cfg,
    task_cfg,
)
sample = next(iter(data_loader))
```

We owe a huge thank you to Peter Wyatt, Kate Tasker, Rachel Taketa, Ali Furkan Biten, Ruben Tito, and their colleagues for their contributions. Their work putting these datasets together has been invaluable. 🤗

Looking Ahead:

We're on a mission to enhance document AI capabilities, and these datasets are just the beginning. With your engagement and innovation, we're confident in the community's ability to develop robust OCR solutions. We encourage you to explore these datasets, experiment with the code, and contribute to the collective progress in document AI.

For detailed information on usage and licensing, please refer to the dataset cards on the Hugging Face hub.

abidlabs VictorSanh lhoestq Dlbk thomwolf manishiitg clefourrier IlyasMoutawwakil osanseviero samusenps DmitryRyumin ajibawa-2023 Pclanglais loubnabnl rwightman marioparreno Chunte NeuralNovel AIAdventurist JoshuaGu antfraia mmhamdy callmeraxz qinluo pankaj-munde Zyn123 andrewrreed ABX-AI pcuenq giux78 victor clem tonnirobo mohammedbriman Wauplin julien-c lunarflu Tom-Neverwinter nikgr kramp francisrafal MoritzLaurer aceeee

rwightman abidlabs VictorSanh lhoestq Dlbk thomwolf IlyasMoutawwakil osanseviero loubnabnl Chunte mmhamdy andrewrreed ABX-AI pcuenq clem Wauplin julien-c lunarflu Tom-Neverwinter

rwightman abidlabs VictorSanh lhoestq Dlbk thomwolf IlyasMoutawwakil osanseviero loubnabnl Chunte andrewrreed ABX-AI clem Jakaline Wauplin julien-c lunarflu

lhoestq Dlbk thomwolf IlyasMoutawwakil osanseviero Chunte femboysLover andrewrreed ABX-AI Wauplin julien-c lunarflu prithivMLmods

shiv2050 Chunte ssml2050 visisco andrewrreed ABX-AI Wauplin julien-c lunarflu Tom-Neverwinter

samusenps Chunte andrewrreed ABX-AI Wauplin julien-c lunarflu Tom-Neverwinter Tonic


Diaries of Open Source. Part 12 🤗

🚀Alibaba releases Qwen1.5-MoE-A2.7B, an interesting MoE with 2.7B activated parameters and 64 experts
Blog https://qwenlm.github.io/blog/qwen-moe/
Demo: https://hf.co/spaces/Qwen/qwen1.5-MoE-A2.7B-Chat-demo
Models: https://hf.co/Qwen
GitHub: https://github.com/QwenLM/Qwen1.5

🎵VoiceCraft, SOTA speech editing and text to speech
GitHub: https://github.com/jasonppy/VoiceCraft
Model: https://huggingface.co/pyp1/VoiceCraft

🐍 AI21Labs release Jamba, an SSM-Transformer, pretrained MoE which allows a large context window (256K) and high throughput
Blog https://www.ai21.com/blog/announcing-jamba
Model https://huggingface.co/ai21labs/Jamba-v0.1

✨ Berkeley releases Starling-LM-7B, an RLHF-ed model, and -RM-34B, a Yi-based reward model very good for its size
Starling Beta: https://hf.co/Nexusflow/Starling-LM-7B-beta
Starling RM: https://hf.co/Nexusflow/Starling-RM-34B

🖥Stability releases Stable Code Instruct 3B, an instruct model for code generation
Blog: https://stability.ai/news/introducing-stable-code-instruct-3b
Demo: https://hf.co/spaces/stabilityai/stable-code-instruct-3b
Report: https://stability.ai/s/Stable_Code_TechReport_release.pdf

📚Common Corpus: the largest public domain dataset for training LLMs
Blog: https://hf.co/blog/Pclanglais/common-corpus
Dataset: https://hf.co/collections/PleIAs/common-corpus-65d46e3ea3980fdcd66a5613

Misc:
⚡GaLore: a very memory-efficient technique that allows pretraining models in consumer GPUs https://hf.co/blog/galore
Moirai
📈Moirai, foundation models for time series forecasting https://hf.co/collections/Salesforce/moirai-10-r-models-65c8d3a94c51428c300e0742
🔥 Mistral-ORPO-Capybara-7K, a high-quality Mistral fine-tune using ORPO, a new alignment technique https://hf.co/kaist-ai/mistral-orpo-capybara-7k
🤯APISR, an anime super-resolution upscaling model https://hf.co/spaces/HikariDawn/APISR

samusenps Dlbk manishiitg Joseph717171 DmitryRyumin ajibawa-2023 HikariDawn ElenaRyumina Chunte gabrielmbmb mmhamdy callmeraxz clem lunarflu nikgr mlabonne JustinLin610

samusenps Dlbk Joseph717171 HikariDawn Chunte mmhamdy clem Titus-von-Koeller lunarflu JustinLin610


Happy to share Living Images and the demo video of the product outpainting model behind it 🚀

Send your generation requests in the thread 🧵or use https://img.coframe.ai

AlekseyKorshuk Glavin001 tunahfish onum TamasZumpf clem lunarflu nikgr qyxs


Fun fact about evaluation!

Did you know that, if you evaluate the same model, with the same prompt formatting & the same fixed few-shot examples, only changing
♻the order in which the few shot examples are added to the prompt ♻
you get a difference of up to 3 points in evaluation score?

I did a small experiment using some MMLU subsets on the best performing 7B and lower pretrained models from the leaderboard.

I tried 8 different prompting methods (containing more or less information, such as just the question, or Question: question, or Question: question Choices: ..., see the x axis) that are commonly used in evaluation.

I then compared the results for all these methods, in 5-shot, during 2 runs. The *only difference* between the first and second run being that the samples used in few-shot are not introduced in the same order.
For example, run one would have been "A B C D E Current sample", vs, in run 2, "D C E A B Current sample".
All the other experiment parameters stayed exactly the same.

As you can see on the attached picture, you get a difference of up to 3 points between the 2 few-shot samples shuffling.

So, when just changing *the order of the few shot samples* can change your results by several points, what is the impact of all other "minimal" and unreported prompting changes?

-> Any kind of model score, provided without an evaluation script for reproducibility, is basically bullshit (or coms).
-> This is why we need reproducible evaluation in a fair and exactly similar setup, using evaluation suites such as `lm_eval` from the Harness, `lighteval` from HF, or the Open LLM Leaderboard.

shiv2050 ssml2050 InferenceIllusionist mmhamdy ricfergas giux78 clem lunarflu nikgr


Teraflop AI is excited to help support the Caselaw Access Project and Harvard Library Innovation Lab, in the release of over 6.6 million state and federal court decisions published throughout U.S. history. It is important to democratize fair access to data to the public, legal community, and researchers. This is a processed and cleaned version of the original CAP data.

During the digitization of these texts, there were erroneous OCR errors that occurred. We worked to post-process each of the texts for model training to fix encoding, normalization, repetition, redundancy, parsing, and formatting.

Teraflop AI’s data engine allows for the massively parallel processing of web-scale datasets into cleaned text form.

Link to the processed dataset: https://huggingface.co/datasets/TeraflopAI/Caselaw_Access_Project

The Caselaw Access Project dataset is licensed under the CC0 License.

We plan to release trillions of commercially licensed text tokens, images, audio, videos, and other datasets spanning numerous domains and modalities over the next months. If you are interested in contributing commercially licensed data be sure to reach out: https://twitter.com/EnricoShippole

Follow us for the next collaborative dataset releases: https://twitter.com/TeraflopAI

osanseviero samusenps rsamantha Pclanglais muhtasham ssml2050 clem nikgr


Hey, I just added three useful advanced use cases to https://huggingface.co/datasets/do-me/SemanticFinder/blob/main/README.md#advanced-use-cases.
SemanticFinder is a collection of embeddings for public documents or books. You can create your own index file from any text or pdf and save it without installing or downloading anything. Try yourself:

1. Translating from 100+ languages to English (even though it might confuse a strawberry with a grapefruit ;D): https://do-me.github.io/SemanticFinder/?hf=List_of_the_Most_Common_English_Words_70320cde&firstOnly=true&inferencingActive=False
2. Finding English synonyms: https://do-me.github.io/SemanticFinder/?hf=List_of_the_Most_Common_English_Words_0d1e28dc&firstOnly=true&inferencingActive=False
3. The "universal index idea": create an embedding index with 30k English words and reuse it on unseen texts. You can decide to fill the gaps in the index by additional inferencing or just stick to the 30k index for instant semantic similarity.
Initial idea: https://github.com/do-me/SemanticFinder/discussions/48
Try here: https://do-me.github.io/SemanticFinder/?hf=List_of_the_Most_Common_English_Words_0d1e28dc&inferencingActive=False&universalIndexSettingsWordLevel with a text of your choice.

This could be enhanced by adding duplets or triplets like "climate change" or "green house gas". Eventually I'd like to set up vector DB integrations.

Super happy to hear your feedback, ideas and maybe even contributions! :)

---
Edit: Apparently markdown url formatting does only work for HF links.


CoDA: Instructive Chain-of-Domain Adaptation with Severity-Aware Visual
  Prompt Tuning (2403.17369)

☁☔ New Research Alert! ❄🌙
📄 Title: CoDA: Instructive Chain-of-Domain Adaptation with Severity-Aware Visual Prompt Tuning

📝 Description: CoDA is a UDA methodology that boosts models to understand all adverse scenes (☁,☔,❄,🌙) by highlighting the discrepancies within these scenes. CoDA achieves state-of-the-art performances on widely used benchmarks.

👥 Authors: Ziyang Gong, Fuhao Li, Yupeng Deng, Deblina Bhattacharjee, Xiangwei Zhu, Zhenming Ji

🔗 Paper: https://huggingface.co/papers/2403.17369

📁 Repository: https://github.com/Cuzyoung/CoDA

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🔍 Keywords: #CoDA #DomainAdaptation #VisualPromptTuning #SAVPT #DeepLearning #Innovation

DmitryRyumin Cusyoung reach-vb osanseviero samusenps shiv2050 ssml2050 Makya

DmitryRyumin nikgr reach-vb osanseviero Cusyoung dimbyTa Makya


[Teaser]: We will post the T2I-Scorer series on HF, a sub project of the Q-Future (esp., Q-Align) that provide a family of more accurate evaluators for T2I generation! Stay tuned for our updates!


Are you looking for the perfect leaderboard/arena for your use case? 👀

There's a new tool for this!
https://huggingface.co/spaces/leaderboards/LeaderboardFinder

Select your modality, language, task... then search! 🔍
Some categories of interest:
- does the leaderboard accept submissions?
- is the test set private or public?
- is it using an automatic metric, human evaluators, or llm as a judge?

The spaces list is build from space metadata, and reloaded every hour.

Enjoy!

m-ric nikgr VictorSanh osanseviero bmorphism


Excited to introduce Jamba by AI21
https://huggingface.co/ai21labs/Jamba-v0.1

We are thrilled to announce Jamba, the world’s first production-grade Mamba based model.

Key Features:
- First production-grade Mamba based model built on a novel SSM-Transformer hybrid architecture
- 3X throughput on long contexts compared to Mixtral 8x7B
- Democratizes access to a massive 256K context window
- The only model in its size class that fits up to 140K context on a single GPU

Jamba is based on a novel architecture that combines Mamba and Transformer. While our initial results show great efficiency gains, we expect this to be further explored and improved with the help of the community.

Check out our blog post for more info: https://ai21-labs.webflow.io/blog/announcing-jamba

osanseviero nikgr taufiqdp samusenps ajibawa-2023 Srulikbdd theKibster victor VictorSanh reach-vb apollomarv Asaf-Yehudai MaziyarPanahi DataSoul callmeraxz tatakof

osanseviero samusenps aswathitsme Srulikbdd reach-vb MaziyarPanahi

osanseviero sadafwalliyani reach-vb wenbopan

sadafwalliyani shiv2050 wenbopan DataSoul


Long-form factuality in large language models (2403.18802)

A new paper titled "Long-Form Factuality in Large Language Models" proposes a new approach to evaluate the long-form factuality of large language models using an AI agent! They introduce SAFE (Search-Augmented Factuality Evaluator) which leverages an LLM to break down responses into individual facts, query Google to verify each fact, and perform multi-step reasoning.

Keypoints:
* SAFE (Search-Augmented Factuality Evaluator) is an automated method using an LLM agent to evaluate factuality
* It also introduces LongFact, a 2,280 prompt set spanning 38 topics to test open-domain factual knowledge
* SAFE achieves a 72% humans agreement while being 20x cheaper. It also wins 76% of the disagreements measured on a small scale experiment where a more thorough human procedure (researchers + full internet search) was used.
* Larger models like GPT-4, Claude Opus and Gemini Ultra tend to exhibit better long-form factuality.

Paper: https://huggingface.co/papers/2403.18802
Code and data: https://github.com/google-deepmind/long-form-factuality

Congrats to the authors for their work!


A Little guide to building Large Language Models in 2024

This is a post-recording of a 75min lecture I gave two weeks ago on how to train a LLM from scratch in 2024. I tried to keep it short and comprehensive – focusing on concepts that are crucial for training good LLM but often hidden in tech reports.

In the lecture, I introduce the students to all the important concepts/tools/techniques for training good performance LLM:
* finding, preparing and evaluating web scale data
* understanding model parallelism and efficient training
* fine-tuning/aligning models
* fast inference

There is of course many things and details missing and that I should have added to it, don't hesitate to tell me you're most frustrating omission and I'll add it in a future part. In particular I think I'll add more focus on how to filter topics well and extensively and maybe more practical anecdotes and details.

Now that I recorded it I've been thinking this could be part 1 of a two-parts series with a 2nd fully hands-on video on how to run all these steps with some libraries and recipes we've released recently at HF around LLM training (and could be easily adapted to your other framework anyway):
*`datatrove` for all things web-scale data preparation: https://github.com/huggingface/datatrove
*`nanotron` for lightweight 4D parallelism LLM training: https://github.com/huggingface/nanotron
*`lighteval` for in-training fast parallel LLM evaluations: https://github.com/huggingface/lighteval

Here is the link to watch the lecture on Youtube: https://www.youtube.com/watch?v=2-SPH9hIKT8
And here is the link to the Google slides: https://docs.google.com/presentation/d/1IkzESdOwdmwvPxIELYJi8--K3EZ98_cL6c5ZcLKSyVg/edit#slide=id.p

Enjoy and happy to hear feedback on it and what to add, correct, extend in a second part.

clem osanseviero giux78 samusenps dillfrescott mohammedbriman reach-vb rwightman Venkman42 gusthema 9bow amenur lisuizhe The-Great-Genius danja drFarid MicheleStingo chkla Skier8402 Artples alemiaschi alirezamsh catastropiyush dlicari iproskurina vikas mkhalifa mertege ianlee1996 LiuPengNGP salkhon jdzw2014 serdarcaglar ma7583 derek-thomas fpadovani zackli4ai Davidqian123


𝐓𝐡𝐞 𝐫𝐞𝐭𝐮𝐫𝐧 𝐨𝐟 𝐭𝐡𝐞 𝐑𝐍𝐍𝐬 ⚔ 𝐍𝐞𝐰 𝐌𝐚𝐦𝐛𝐚-𝐛𝐚𝐬𝐞𝐝 𝐚𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 "𝐉𝐚𝐦𝐛𝐚"

Since the release of BERT by Google in 2019, Transformers architecture have taken over machine learning thanks to their 𝗮𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗺𝗲𝗰𝗵𝗮𝗻𝗶𝘀𝗺, that gives them the ability to focus on important points of the input. But 𝙖𝙩𝙩𝙚𝙣𝙩𝙞𝙤𝙣 𝙘𝙤𝙢𝙥𝙪𝙩𝙖𝙩𝙞𝙤𝙣 𝙞𝙨 𝙦𝙪𝙖𝙙𝙧𝙖𝙩𝙞𝙘 𝙞𝙣 𝙩𝙝𝙚 𝙞𝙣𝙥𝙪𝙩 𝙡𝙚𝙣𝙜𝙩𝙝.

💫 The Mamba paper, published in December 2023, announced the return of the RNNs: it has no attention, but integrates a selection mechanism, which should be able to reproduce the “focus” ability of attention, in an architecture for which the compute requirements 𝗴𝗿𝗼𝘄 𝗼𝗻𝗹𝘆 𝗹𝗶𝗻𝗲𝗮𝗿𝗹𝘆 𝗶𝗻 𝗶𝗻𝗽𝘂𝘁 𝗹𝗲𝗻𝗴𝘁𝗵!
🤔 Would this work? We had yet to see a large Mamba model recovering the performance of Attention-based Transformers.

💥 But now it's done! A (Mamba + Transformers) hybrid just beat Transformers!

The AI21 Labs team just released Jamba.
They insert a few Transformer layers to inject some attention in a big pile of Mamba layers, thus getting the best of both worlds.

𝙏𝙇;𝘿𝙍:
🏗 𝗡𝗲𝘄 𝗠𝗼𝗘 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲: 4 Jamba blocks, each of these being 7 Mamba layers for 1 Transformer.
🏋 𝟱𝟮𝗕 𝗽𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿𝘀, 𝟭𝟮𝗕 𝗮𝗰𝘁𝗶𝘃𝗲 𝗮𝘁 𝗶𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲: This reduction is enabled by Mixture of Experts, and similar to Mixtral (47B parameters - 13B active).
🏎 𝗦𝗽𝗲𝗲𝗱: 𝘅𝟯 𝘁𝗵𝗿𝗼𝘂𝗴𝗵𝗽𝘂𝘁. Jamba is much faster than similar-sized Transformer models on long contexts.
📏 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗹𝗲𝗻𝗴𝘁𝗵: 𝟭𝟰𝟬𝗞 𝘁𝗼𝗸𝗲𝗻𝘀 on a single 80GB A100!
💪 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲: 𝘀𝘁𝗮𝘁𝗲-𝗼𝗳-𝘁𝗵𝗲-𝗮𝗿𝘁 𝗳𝗼𝗿 𝘁𝗵𝗶𝘀 𝘀𝗶𝘇𝗲. The small injection of attention seems sufficient since Jamba beats the open-source reference Mixtral-8x7B on many benchmarks!

Try it here 👉 https://huggingface.co/ai21labs/Jamba-v0.1

monsoon-nlp lunarflu carbonox-infernox dhuynh95 osanseviero vonehrenheim nikgr samusenps Amano13 reach-vb Chunte seyf1elislam blanchon


Just now, we release a small MoE model, Qwen1.5-MoE-A2.7B, a 14B model with 2.7B activated parameters. Leaving the hype, I would love to share more things here in HF. But if you don't know much about this, check our blog for more info: https://qwenlm.github.io/blog/qwen-moe/

At the beginning, it was trying with the MoE stuff, making Megatron work well with MegaBlocks. As always, we worked with small ones first. However, we have been struggling with a lot of details.

With megablocks and so many tricks that make training MoE models work, it is almost impossible to fail. The challenge is actually how good your model is. Then things became more complex than I had expected. Finegrained experts actually pissed me off but damn it works for the model at this scale. However, it brings complexity to the model, and this is somehow why at this moment our codes are not merged into llama.cpp cuz it really brings problems. Shared experts might be good, but we need more engineering efforts to really unleash its benefits in inference acceleration.

For the community, this is actually our first time releasing an MoE model. We don't know what will happen to us, but we are prepared for complaints. I just hope that we can really make things clear, and provide a good recipe to play with our MoE model just like people playing with Mixtral.

Nbardy lunarflu CulturedMan osanseviero nikgr manishiitg reach-vb pierrci aguspiza xiaoniqiu alielfilali01 Shuaiii

andysalerno osanseviero Fizzarolli samusenps reach-vb alielfilali01 btcnhung1299 Shuaiii


 🎈 LLM Benchmarks Update!

**tl;dr: do not depend on benchmark leaderboards to choose your "chatbot" model! (Especially for non-English languages.)**

First of all, I'm discontinuing the Open #Dutch #LLM Leaderboard (https://lnkd.in/eFnsaFR6). It will stay online for now, but I urge the use of the ScandEval leaderboard instead (https://scandeval.com/dutch-nlg/) by @saattrupdan. It contains more tasks, has better reproducibility and statistics (CI) and a flexible back-end library (`scandeval`) to run your own benchmarks with. As part of project "Leesplank" (with Michiel Buisman and Maarten Lens-FitzGerald) we recently added GPT-4-1106-preview scores to add a good "target" to the leaderboard.

An important note here is that benchmark leaderboards are not a golden truth. Especially evaluating generative models is hard. You run into issues like prompt engineering (and sensitivity of models to one or other prompt), structured output generation, and - quite simply - "how to automatically evaluate open-ended generation".

💡 Another important but under-discussed facet is the discrepancy between models' capability of understanding vs. generating *in different languages* (so the NLU part of NLG benchmarking). In other words: some of the listed models score really well on, e.g., MCQ benchmarks but are not suitable to use as DUTCH chat bots. Interestingly, some of these models seem to understand questions in Dutch and are able to pick the right answer (because they have good knowledge or reasoning skills), but generating fluent and grammatical Dutch is something else entirely! This is perhaps also true for humans: it's easier to sort-of grasp the meaning of a new language and answer with "Yes" or "No", but answering fluently in the language is much harder! Yet, your language production fluency does not necessarily say anything about your knowledge and reasoning skills.

Hopefully we can get a chat arena for Dutch some day - user feedback is the most powerful metric!

davanstrien lunarflu nikgr saattrupdan reach-vb shiv2050 ssml2050

Rijgersberg osanseviero lunarflu robinsmits reach-vb


Mini-Gemini: Mining the Potential of Multi-modality Vision Language
  Models (2403.18814)

VILA: On Pre-training for Visual Language Models (2312.07533)

FeatUp: A Model-Agnostic Framework for Features at Any Resolution (2403.10516)

Diaries of Open Source. Part 11 🚀

🚀Databricks release DBRX, potentially the best open access model! A 132B Mixture of Experts with 36B active params and trained on 12 trillion tokens
Blog: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm
Base and instruct models: https://hf.co/collections/databricks/dbrx-6601c0852a0cdd3c59f71962
Demo: https://hf.co/spaces/databricks/dbrx-instruct

🤏1-bit and 2-bit quantization exploration using HQQ+
Blog post: https://mobiusml.github.io/1bit_blog/
Models: https://hf.co/collections/mobiuslabsgmbh/llama2-7b-hqq-6604257a96fc8b9c4e13e0fe
GitHub: https://github.com/mobiusml/hqq

📚Cosmopedia: a large-scale synthetic dataset for pre-training - it includes 25 billion tokens and 30 million files
Dataset: https://hf.co/datasets/HuggingFaceTB/cosmopedia
Blog: https://hf.co/blog/cosmopedia

⭐Mini-Gemini: multi-modal VLMs, from 2B to 34B
Models: https://hf.co/collections/YanweiLi/mini-gemini-6603c50b9b43d044171d0854
Paper: https://huggingface.co/papers/2403.18814
GitHub: https://github.com/dvlab-research/MiniGemini

🔥VILA - On Pre-training for VLMs
Models: https://hf.co/collections/Efficient-Large-Model/vila-on-pre-training-for-visual-language-models-65d8022a3a52cd9bcd62698e
Paper: https://hf.co/papers/2312.07533

Misc
👀 FeatUp: a framework for image features at any resolution: https://hf.co/spaces/mhamilton723/FeatUp https://hf.co/papers/2403.10516
🍞ColBERTus Maxiums, a colbertialized embedding model https://hf.co/mixedbread-ai/mxbai-colbert-large-v1
🖌Semantic Palette, a new drawing paradigm  https://hf.co/spaces/ironjr/SemanticPalette
🧑⚕HistoGPT, a vision model that generates accurate pathology reports  https://hf.co/marr-peng-lab/histogpt https://www.medrxiv.org/content/10.1101/2024.03.15.24304211v1

mobicham appoose loubnabnl davanstrien samusenps DmitryRyumin ajibawa-2023 clem GO4code victor vladbogo nikgr VictorSanh reach-vb manuel-tran

ironjr loubnabnl samusenps clem reach-vb manuel-tran


ViTAR: Vision Transformer with Any Resolution (2403.18361)

🎉 🎉 🎉 Happy to share our recent work. We noticed that image resolution plays an important role, either in improving multi-modal large language models (MLLM) performance or in Sora style any resolution encoder decoder, we hope this work can help lift restriction of 224x224 resolution limit in ViT.

https://huggingface.co/papers/2403.18361

ajibawa-2023 samusenps DmitryRyumin clem lunarflu victor nikgr reach-vb shiv2050 ak0601 ncoop57 talaviyabhavik


Compared Effect Of Image Captioning For SDXL Fine-tuning / DreamBooth Training for a Single Person, 10.3 GB VRAM via OneTrainer

Sadly the post character count is limited so please read full info on Medium here

https://medium.com/@furkangozukara/compared-effect-of-image-captioning-for-sdxl-fine-tuning-dreambooth-training-for-a-single-person-961087e42334


AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation (2403.17694)

🚀🎭🌟 New Research Alert! 🌟 🎭🚀
📄 Title: AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation 🔝

📝 Description: AniPortrait is a novel framework for generating photorealistic portrait animations driven by audio and a reference image, with superior facial naturalness, pose variety, and visual quality, with potential applications in facial motion editing and facial reenactment.

👥 Authors: Huawei Wei, @ZJYang, Zhisheng Wang

🔗 Paper: https://huggingface.co/papers/2403.17694

📁 Repository: https://github.com/Zejun-Yang/AniPortrait

🤗 Demo: https://huggingface.co/spaces/ZJYang/AniPortrait_official
🔥 Model 🤖: https://huggingface.co/ZJYang/AniPortrait

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords:  #AniPortrait #Animation #AudioDriven #Photorealistic #FacialAnimation #DeepLearning #Innovation

DmitryRyumin hinach4n samusenps MiSTe-R clem lunarflu victor reach-vb Poulain


The Unreasonable Ineffectiveness of the Deeper Layers (2403.17887)

The Unreasonable Ineffectiveness of the Deeper Layers

https://huggingface.co/papers/2403.17887

We empirically study a simple layer-pruning strategy for popular families of open-weight pretrained LLMs, finding minimal degradation of performance on different question-answering benchmarks until after a large fraction (up to half) of the layers are removed. To prune these models, we identify the optimal block of layers to prune by considering similarity across layers; then, to "heal" the damage, we perform a small amount of finetuning. In particular, we use parameter-efficient finetuning (PEFT) methods, specifically quantization and Low Rank Adapters (QLoRA), such that each of our experiments can be performed on a single A100 GPU. From a practical perspective, these results suggest that layer pruning methods can complement other PEFT strategies to further reduce computational resources of finetuning on the one hand, and can improve the memory and latency of inference on the other hand. From a scientific perspective, the robustness of these LLMs to the deletion of layers implies either that current pretraining methods are not properly leveraging the parameters in the deeper layers of the network or that the shallow layers play a critical role in storing knowledge.

samusenps ajibawa-2023 osanseviero clem ishmnnit Aryanne KvrParaskevi


SegGPT is a vision generalist on image segmentation, quite like GPTs for computer vision ✨
It comes with the last release of transformers 🎁  Demo and more in this post!
SegGPT is an extension of the Painter, where you speak to images with images: the model takes in an image prompt, transformed version of the image prompt, the actual image you want to see the same transform, and expected to output the transformed image.
SegGPT consists of a vanilla ViT with a decoder on top (linear, conv, linear).
The model is trained on diverse segmentation examples, where they provide example image-mask pairs, the actual input to be segmented, and the decoder head learns to reconstruct the mask output.
This generalizes pretty well!
The authors do not claim state-of-the-art results as the model is mainly used zero-shot and few-shot inference. They also do prompt tuning, where they freeze the parameters of the model and only optimize the image tensor (the input context).
Thanks to 🤗 transformers you can use this model easily!
See here https://huggingface.co/docs/transformers/en/model_doc/seggpt
I have built an app for you to try it out. I combined SegGPT with Depth Anything Model, so you don't have to upload image mask prompts in your prompt pair 🤗
Try it here https://huggingface.co/spaces/merve/seggpt-depth-anything
Also check out the collection https://huggingface.co/collections/merve/seggpt-660466a303bc3cd7559d271b

samusenps osanseviero clem lunarflu talaviyabhavik


Thanks to the 4000+ attempts to jailbreak LLM chatbots and 2000+ votes cast on the Chatbot Guardrails Arena in the 4 days since release!

Based on the players' feedback, we have updated the instructions to be clearer and to emphasize that players should vote after identifying a secure chatbot.

Tell us what you think when you play again: https://huggingface.co/spaces/lighthouzai/guardrails-arena

samusenps osanseviero Tom-Neverwinter clefourrier clem lunarflu reach-vb AtAndDev normanschizogh chuangxinlezhi sbrandeis

samusenps osanseviero Tom-Neverwinter clem lunarflu reach-vb AtAndDev

samusenps Tom-Neverwinter clem lunarflu reach-vb AtAndDev xpgx1


Here, we share the technical details of InternLM2, the current state-of-the-art open-source LLMs!!

See collections https://huggingface.co/collections/internlm/internlm2-65b0ce04970888799707893c
Paper: https://huggingface.co/papers/2403.17297

samusenps osanseviero clem bedirt victor


Have we really squeezed out the capacity of a compact chat model? Thrilled to see our latest open model, Starling-7B, ranks 13th among all models in Chatbot Arena!
🚀 As a 7B model, Starling surpasses larger open and proprietary models, including Claude-2, GPT-3.5-Turbo, Gemini Pro, Mixtral 8x7B and Llama2-70B, and is currently the best 7B chat model in Chatbot Arena!
Try out the model on HF here: https://huggingface.co/Nexusflow/Starling-LM-7B-beta

brian-yu-nexusflow banghua clem ThWu jian-zhang-nexusflow mahiatlinux osanseviero unnaymed Lewdiculous dalraf lunarflu mohammedbriman ucyang

banghua brian-yu-nexusflow damon-mosk-aoyama-nexusflow clem ThWu jian-zhang-nexusflow mahiatlinux coolkrishds osanseviero natolambert lunarflu

banghua brian-yu-nexusflow samusenps clem ThWu jian-zhang-nexusflow mahiatlinux osanseviero dalraf lunarflu

brian-yu-nexusflow banghua ThWu jian-zhang-nexusflow mahiatlinux lunarflu

banghua ThWu jian-zhang-nexusflow mahiatlinux lunarflu brian-yu-nexusflow


New state-of-the-art open LLM! 🚀 Databricks just released DBRX, a 132B MoE trained on 12T tokens. Claiming to surpass OpenAI GPT-3.5 and is competitive with Google Gemini 1.0 Pro. 🤯

TL;DR
🧮 132B MoE with 16 experts with 4 active in generation
🪟 32 000 context window
📈 Outperforms open LLMs on common benchmarks, including MMLU
🚀 Up to 2x faster inference than Llama 2 70B
💻 Trained on 12T tokens
🔡 Uses the GPT-4 tokenizer
📜 Custom License, commercially useable

Collection: https://huggingface.co/collections/databricks/dbrx-6601c0852a0cdd3c59f71962
Demo: https://huggingface.co/spaces/databricks/dbrx-instruct

Kudos to the Team at Databricks and MosaicML for this strong release in the open community! 🤗

victor alielfilali01 Shinku osanseviero samusenps giux78 Tom-Neverwinter mmhamdy sepal giulianobr carlos-eduardo eppie2 VictorSanh dalraf lunarflu thomwolf clem KvrParaskevi dagelf flymonk

clem Tom-Neverwinter mmhamdy eppie2 dalraf lunarflu mohammedbriman matthewgjohnson


 🐰 Happy ML Easter! 🐰

I recently had a wager with my colleagues which had me create AI-assisted videos of myself in an Easter Bunny costume singing an AI-generated easter song (in different languages).

I did it in 3 simple steps:
🐇I created an easter Bunny version of myself with this space: https://huggingface.co/spaces/multimodalart/Ip-Adapter-FaceID by @multimodalart
🎶 Created a song on suno.ai
🎞 Used the Dreamtalk space by @fffiloni to animate the image, conditioned on the song: https://huggingface.co/spaces/fffiloni/dreamtalk

fffiloni osanseviero clem victor linoyts Wauplin kramp yyassif MarinaraSpaghetti sayandafadar alielfilali01 sbrandeis


Would 1-2 sentence tl;dr summaries of datasets on the Hub be useful for you?

For example, for the https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T dataset, would the following summary help give you a quick sense of its content?

> tl;dr: RedPajama is a fully open-source implementation of the LLaMa dataset, consisting of 1.2 trillion tokens from sources like Commoncrawl, C4, GitHub, Books, ArXiv, Wikipedia, and StackExchange, primarily in English, and is structured with metadata for each text sample.

I've created a dataset with example summaries of the 500 most liked datasets on the Hub: https://huggingface.co/datasets/davanstrien/dataset-tldr

Would these kinds of summaries be helpful?

osanseviero merve clem samusenps sequelbox DataSoul Arakinas


Major TOM: Expandable Datasets for Earth Observation (2402.12095)

𝗠𝗮𝗷𝗼𝗿 𝗧𝗢𝗠: 𝗣𝗹𝗮𝗻𝗲𝘁 𝗘𝗮𝗿𝘁𝗵 𝗶𝘀 𝗯̶𝗹̶𝘂̶𝗲̶ 𝟱.𝟰𝟬𝟱 𝗚𝗛𝘇

🚨 EXPANSION RELEASE: 𝗦𝗲𝗻𝘁𝗶𝗻𝗲𝗹-𝟭 𝗶𝘀 𝗻𝗼𝘄 𝗮𝘃𝗮𝗶𝗹𝗮𝗯𝗹𝗲 in the MajorTOM-Core!
https://huggingface.co/datasets/Major-TOM/Core-S1RTC

🏎 Together with @aliFrancis we've been racing to release the first official expansion to the Major TOM project.

MajorTOM-Core-S1RTC contains 1,469,955 of SAR images paired to Sentinel-2 images from Core-S2.

🔍We cover more than 65% of the optical coverage with an average time shift of 7 days.

16 TB of radiometrically calibrated SAR imagery, available in the exact same format as the existing Major-TOM data.

🗺 You can explore instantly in our viewing app:
https://huggingface.co/spaces/Major-TOM/MajorTOM-Core-Viewer

So, what now?

🧱 𝐂𝐨𝐦𝐦𝐮𝐧𝐢𝐭𝐲 𝐆𝐫𝐨𝐰𝐭𝐡: our community continues to grow! To coordinate the upcoming expansions as well as use cases of the open data, we will organise a meet up on 23 April, you can 𝐫𝐞𝐠𝐢𝐬𝐭𝐞𝐫 𝐲𝐨𝐮𝐫 𝐢𝐧𝐭𝐞𝐫𝐞𝐬𝐭 here: https://forms.gle/eBj8JvibJx9b6PLf9

🚂 𝐎𝐩𝐞𝐧 𝐃𝐚𝐭𝐚 𝐟𝐨𝐫 𝐎𝐩𝐞𝐧 𝐌𝐨𝐝𝐞𝐥𝐬: Major-TOM Core dataset is currently supporting several strands of ongoing research within and outwith our lab and we are looking forward to the time when we can release models that take advantage of that data! https://huggingface.co/Major-TOM

📌 𝐏𝐨𝐬𝐭𝐞𝐫 𝐚𝐭 𝐈𝐆𝐀𝐑𝐒𝐒: We will present Major TOM project as a poster at IGARSS in Athens (July) - come talk to us if you're there! You can access the paper here: https://huggingface.co/papers/2402.12095

🌌 Developed at European Space Agency Φ-lab in partnership with Hugging Face

aliFrancis davanstrien osanseviero clem Tom-Neverwinter robmarkcole


Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding
  Model Mechanisms (2403.17806)

🔍 Today's pick in Interpretability & Analysis of LMs: Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms by @mwhanna @sandropezzelle @belinkov

Edge attribution patching (EAP) is a circuit discovery technique using gradients to approximate the effects of causal intervening on each model edge. In the literature, its effectiveness is validated by comparing the overlap of its resulting circuits with those found via causal interventions (much more expensive).

This work:

1. Proposes a new method for faithful and efficient circuit discovery named edge attribution patching with integrated gradients (EAP-IG)
2. Evaluates the faithfulness EAP, EAP-IG and activation patching, i.e. whether behavior of the model remains consistent after all non-circuit edges are ablated.
3. Highlights that, while the no-overlap and full-overlap of EAP-like methods with activation patching results are generally good indicators of unfaithful and faithful (respectively) circuit identification, circuits with moderate overlap cannot generally assumed to be faithful to model behavior.

An advantage of EAP-IG is enabling the usage of KL-Divergence as a target for gradient propagation, which is not possible in the case of raw gradient-based EAP.

EAP-IG runtime is approximately similar to the one of EAP, with a small number of steps to approximate the gradient integral.

Importantly, circuit faithfulness does not imply completeness, i.e. whether all components participating towards a specific task were accounted for. This aspect is identified as interesting for future work.

📄 Paper: https://huggingface.co/papers/2403.17806

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9

javifer osanseviero samusenps clem giux78 mmhamdy alemiaschi