CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding
  Benchmark (2401.11944)

CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding Benchmark

paper page: https://huggingface.co/papers/2401.11944

As the capabilities of large multimodal models (LMMs) continue to advance, evaluating the performance of LMMs emerges as an increasing need. Additionally, there is an even larger gap in evaluating the advanced knowledge and reasoning abilities of LMMs in non-English contexts such as Chinese. We introduce CMMMU, a new Chinese Massive Multi-discipline Multimodal Understanding benchmark designed to evaluate LMMs on tasks demanding college-level subject knowledge and deliberate reasoning in a Chinese context. CMMMU is inspired by and strictly follows the annotation and analysis pattern of MMMU. CMMMU includes 12k manually collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering, like its companion, MMMU. These questions span 30 subjects and comprise 39 highly heterogeneous image types, such as charts, diagrams, maps, tables, music sheets, and chemical structures. CMMMU focuses on complex perception and reasoning with domain-specific knowledge in the Chinese context. We evaluate 11 open-source LLMs and one proprietary GPT-4V(ision). Even GPT-4V only achieves accuracies of 42%, indicating a large space for improvement. CMMMU will boost the community to build the next-generation LMMs towards expert artificial intelligence and promote the democratization of LMMs by providing diverse language contexts.

osanseviero clefourrier lunarflu clem afrideva walzay AdinaY


Universal Neurons in GPT2 Language Models (2401.12181)

🔍 Today's pick in Interpretability & Analysis of LMs: Universal Neurons in GPT2 Language Models by @wesg @thorsley @luminolblue T.R. Kheirkhah Q. Sun, @willhath @NeelNanda D. Bertsimas

This work investigates the universality of individual neurons across GPT2 models trained from different initial random seeds, starting from the assumption that such neurons are likely to exhibit interpretable patterns. The authors find 1-5% of neurons showing high correlation across five model seeds, i.e. consistently activating for the same inputs. In particular, those neurons can be grouped into families exhibiting similar functional roles, e.g. modulating the next token prediction entropy, controlling the output norm of an attention head, and promoting/suppressing vocabulary elements in the prediction. Finally, universal neurons are often observed to form antipodal pairs, conjectured to improve the robustness and calibration of model predictions via ensembling.

📄 Paper: https://huggingface.co/papers/2401.12181

merve osanseviero wesg lunarflu abidlabs afrideva LiraMirui Aryanne sbrandeis

s3nh merve osanseviero wesg clem abidlabs lunarflu


Hi Community!
I'm ecstatic to announce Flutter SDK for HuggingFace Inference APIs. The project is open-source and can be found at https://github.com/shivance/huggingface.dart . I have also created a sample Flutter App that does language translation using HuggingFace's Helsinki-NLP/opus-mt-ru-en" model. Below is the video demo.

## Why Flutter?

Flutter is an open-source UI software development kit created by Google. It is used to develop cross-platform applications from a single codebase for any Web Browser, Android, iOS, Linux, macOS, and Windows.

Over the past few years, the framework has grown a lot of popularity.

## Mission
This is a small step-stone to bring Open Source Machine Learning to the edge and thus make AI more accessible. This project, at the time of writing, supports only NLP APIs, I'll slowly add other modalities too.

samusenps afrideva osanseviero tomaarsen Wauplin merve lunarflu clem hereral kevineen jaecabrera valfcn

samusenps osanseviero merve chansung lunarflu clem


𝗛𝗼𝘄 𝘄𝗲 𝗺𝗮𝗱𝗲 𝗚𝗿𝗮𝗱𝗶𝗼 𝗳𝗮𝘀𝘁𝗲𝗿 𝗯𝘆... 𝘀𝗹𝗼𝘄𝗶𝗻𝗴 𝗶𝘁 𝗱𝗼𝘄𝗻!

About a month ago, @oobabooga (who built the popular text generation webui) reported an interesting issue to the Gradio team. After upgrading to Gradio 4, @oobabooga noticed that chatbots that streamed very quickly had a lag before their text would show up in the Gradio app.

After some investigation, we determined that the Gradio frontend would receive the updates from the backend immediately, but the browser would lag before rendering the changes on the screen. The main difference between Gradio 3 and Gradio 4 was that we migrated the communication protocol between the backend and frontend from Websockets (WS) to Server-Side Events (SSE), but we couldn't figure out why this would affect the browser's ability to render the streaming updates it was receiving.

After diving deep into browsers events, @aliabid94 and @pngwn made a realization: most browsers treat WS events (specifically the `WebSocket.onmessage` function) with a lower priority than SSE events (`EventSource.onmessage` function), which allowed the browser to repaint the window between WS messages. With SSE, the streaming updates would stack up in the browser's event stack and be prioritized over any browser repaint. The browser would eventually clear the stack but it would take some time to go through each update, which produced a lag.

We debated different options, but the solution that we implemented was to introduce throttling: we slowed down how frequently we would push updates to the browser event stack to a maximum rate of 20/sec. Although this seemingly “slowed down” Gradio streaming, it actually would allow browsers to process updates in real-time and provide a much better experience to end users of Gradio apps.

See the PR here: https://github.com/gradio-app/gradio/pull/7084

Kudos to @aliabid94 and @pngwn for the fix, and to @oobabooga and @pseudotensor for helping us test it out!

kramp osanseviero pngwn Ifeanyi xpgx1 BramVanroy takeraparterer samusenps clem phanes gate369 fffiloni sbarman25 taufiqdp akhaliq ysharma davanstrien gsarti lunarflu julien-c merve hysts freddyaboulton

clem akhaliq ysharma kramp santiviquez gsarti lunarflu julien-c

takeraparterer clem akhaliq ysharma gsarti lunarflu julien-c


Re-posting @karpathy's blogpost here because it's down on https://karpathy.github.io/2024/01/21/selfdriving-agi. What do you all think?

elpreneurAbdo RodolpheD Paul-HF SaylorTwift stefan-jo arnaudliotta lunarflu julien-c merve radames pierrci

osanseviero chansung samu lunarflu julien-c merve


Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data (2401.10891)

Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data

paper page: https://huggingface.co/papers/2401.10891
demo: https://huggingface.co/spaces/LiheYoung/Depth-Anything

Depth Anything is trained on 1.5M labeled images and 62M+ unlabeled images jointly, providing the most capable Monocular Depth Estimation (MDE) foundation models with the following features:

zero-shot relative depth estimation, better than MiDaS v3.1 (BEiTL-512)

zero-shot metric depth estimation, better than ZoeDepth

optimal in-domain fine-tuning and evaluation on NYUv2 and KITTI

Norod78 jgitsolutions clem Chunte alazro victor samusenps XaevrM NeuralNovel Kutches


GPU Poor POV: Dont be Afraid :D

Sometimes we dont want to do something because of low self esteem,
I ofter hear 'its to hard for me','i am not an expert','i do not know how to do it', etc. These  words are  never the truth,  we should not be afraid and try to build something because there is no additive value without a failure.

Same things comes in LLMs, there is a lot of fancy words happening, but whats is more important is that there are also  people who are constantly building so other can build.  Diving into finetuning LLMs is incredibly simple if we assume using axolotl library and pretrains stored on huggingface.

All we need is an idea, our GPU Poor desktop or colab notebooks and these steps:
```
git clone https://github.com/OpenAccess-AI-Collective/axolotl
cd axolotl

pip3 install packaging
pip3 install -e '.[flash-attn,deepspeed]'
```
After installation process we can go to examples, and modify configs to our own needs.
Lets jump into
```
axolotl\examples\llama-2\qlora.yml
```
and change
```
base_model: NousResearch/Llama-2-7b-hf
```
to
```
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
```
choose dataset from huge amounts of dataset that are possible to use from hf.co/datasets and tweak additional params like batch_size, number of epochs, how often do we want to save our model and many more (which I wont focus on rn).
Then,
```
accelerate launch -m axolotl.cli.train examples/llama-2/qlora.yml
```
Will allow to start the finetuning process on structure defined strictly by you.  After finetuning, model will be saved in path provided in config, and you can check out if it performs better than  the base one. Or even you can put it on llm Leaderboard to check if we do not have new SOTA :)
Have fun and have a great day <3

KnutJaegersberg Csplk miesnerjacob victor alazro becstar samusenps ameerazam08 NeuralNovel pro7357 s3nh Tanvir1337 Solshine merve aardoi joujiboi danielus Ji-Ha


Towards Best Practices of Activation Patching in Language Models:
  Metrics and Methods (2309.16042)

🔍 Today's pick in Interpretability & Analysis of LMs: Towards Best Practices of Activation Patching in Language Models: Metrics and Methods by @m0pp11 and @NeelNanda

This work systematically examines the impact of methodological details in activation patching, a popular technique with causal guarantees to quantify the importance of model components in driving model predictions. Authors' recommendations include 1) using in-distribution counterfactual prompts instead of noise/zeroing to mitigate the OOD problem, 2) using logins instead of probabilities as evaluation metrics to enable the discovery of model components with negative influence on predictions, 3) accounting for interaction factors across layers when performing multi-layer patching; and 4) experiment with corrupting different prompt tokens to verify their agreement in the resulting discovered circuits.

📄 Paper: https://huggingface.co/papers/2309.16042

clem victor osanseviero lunarflu clefourrier m0pp11


Update on the Newsletter of 🤗 Daily Paper

Automatic Korean translation is integrated. In the newspaper, "KO" links appear, and it will bring you to the translated version of full paper. This is done with the following workflow.

1. Grasp the list of arXiv IDs from 🤗 Daily Paper API
2. Distribute a number of sub-list of arXiv IDs to VMs (possibly spot instances since the job ends shortly)
3. Commit & push the translated paper in HTML to the designated GitHub repository
4. Newsletter will include the links to the HTML of each paper

Job distribution to a number of VMs are super easily done with [dstack]( https://dstack.ai/ ), and the translation sub-workflow is done through 1) download PDF of each paper with arxiv-dl package, 2) PDF => text with nougat-ocr package, 3) a custom trained model( https://huggingface.co/nlp-with-deeplearning/enko-t5-small-v0 ) in 🤗 transformers to translate the English text into Korean line by line, and 4) reformat the translation into HTML.

Many people in Korea are not fluent in English but want to learn about new stuff in AI, so they usually use Google Translate or other services. This is why I made this feature for easier and direct access to the SOTA knowledge.

Are there other countries with the similar needs? If so, it would be wonderful to cooperate to support more languages. Please reach out anyone is interested in this.

PS; I always wanted to show the usefulness of open ML models by building a well working end to end product, and this newsletter shows it by featuring T5ForConditionalGeneration (translation), SOLAR LLM (summarization).

if you want to sub to the newsletter
: https://groups.google.com/g/hf-daily-paper-newsletter

if you want to look into the source codes
: https://github.com/deep-diver/hf-daily-paper-newsletter

s3nh peterschmidt85 aidystark afrideva clem victor samusenps maywell lunarflu julien-c shrijayan ysharma dillfrescott JUNGU


Migrated all my GPU consuming Spaces to ZERO, it was super easy to do so (add three lines of code and voila!) and the start-up time decreased dramatically as well 💜

s3nh NeuralNovel SharathChenna samusenps jkopz akhaliq Chunte chansung aidystark gbharti osanseviero abidlabs clem KnutJaegersberg koisose Tanvir1337 awinml julien-c pierrci clefourrier Norod78 Sentdex mikr cbensimon tsukumijima mahiatlinux asad Ryukijano taesiri Nymbo leeloolee


GPU Poor POV: My storytelling choices of the week

Its end of the week, I decided to summarize my observations in community based LLMs and mention few models in specific area which are very interesting and has capability to create some insightful stories despite of its relatively lightweight form.

I personally did not use LLMs in my daily routine to tasks like function calling, parsing or assist in code writing. What I tried to use for is storytelling, because it always amaze me how different these models comes to different preferred tasks.

How this model are able to generalize the stories and sometimes, how high level of creativity they carry.

https://huggingface.co/BlueNipples/DaringLotus-v2-10.7b its main target is to generate prose. Quoting the author 'It shares it's good prose, and relatively decent coherency, being a little bit more on the side of prose, and a little bit less on the side of coherency. I like this model for generating great prose if I feel like regening a bit. '

https://huggingface.co/NeuralNovel/Aeryth-7B-v0.1
great work by @NeuralNovel , I really like how flexible this model is, there is no strict focus on a certain role, so definitely worth a try. Would love to hear more about dataset on which was trained, afaik is private rn. best suited for Science Fiction, History & Romance genres due to the training data used.

And the last one for today is https://huggingface.co/FPHam/Sydney_Pirate_Mistral_7b @FPHam work always amaze me how the models are able to stick to provided role. awesome work as always, Ill for sure use this model to generate some interesting stories.

I know that hype train is going fast but as I observe people here on huggingface are creating really creative models which are for sure worth to try. Have a great day <3

KnutJaegersberg osanseviero FlareRebellion NeuralNovel Dlbk abdd199719 merve samusenps NovoCode HuggyMonkey dvilasuero abidlabs clem s3nh pro7357 Tanvir1337

NeuralNovel samusenps NovoCode afrideva clem Tanvir1337


I've prepared  a Google Colab notebook  which allows you to play with interpolating between different people using IP-Adapter SDXL Face-ID Plus.

```

#Prepare a list t of num_of_results values between 0 and 1
t_space = torch.linspace(0, 1, num_of_results)
for t in tqdm(t_space):
    mix_factor = t.item()
    # interpolate between the two face images
    image = (image1 * (1 - mix_factor) + image2 * mix_factor).astype(np.uint8)
    # interpolate between the two face embedding
    faceid_embeds = torch.lerp(faceid_embeds1, faceid_embeds2, t)
   #generate interpolated result
    images = ip_model.generate(prompt=prompt, negative_prompt=negative_prompt, face_image=image, faceid_embeds=faceid_embeds, shortcut=v2, num_samples=2, scale=scale, s_scale=s_scale, guidance_scale=guidance_scale, width=width, height=height, num_inference_steps=steps, seed=seed)

```

Link to notebook:
https://colab.research.google.com/#fileId=https%3A//huggingface.co/datasets/Norod78/face_id_v2_test_code/blob/main/norod78_faceid_sdxl_plus_v2_test.ipynb

Link to Face-ID Repo:
https://huggingface.co/h94/IP-Adapter-FaceID

Link to all sorts of generated examples (Use the file tab):
https://huggingface.co/datasets/Norod78/face_id_v2_test_code/tree/main/sdxl_plus_v2_outputs

osanseviero linoyts clem merve HuggyMonkey tonyassi

s3nh clem kevinstonge samusenps tonyassi fahim1233


There's a lot of interest in machine learning models that generate 3D objects, so Gradio now supports previewing STL files natively in the  `Model3D` component. Huge thanks to `Monius` for the contribution 🔥🔥

samusenps Maykeye osanseviero AIIAR Dlbk clem rreed linoyts johko Chunte HuggyMonkey lunarflu ciCic AgastyaPatel marianbasti goendalf666 sugatoray conceptron

samusenps AIIAR clem abidlabs taufiqdp NeuralNovel Chunte lunarflu


🌳 Model Family Tree

Merging models has become a powerful way to compress information and build powerful models for cheap. Right now, the process is still quite experimental: which models to merge? which parameters should I use? We have some intuition but no principled approach.

I made a little tool to make things a little clearer. It allows you to visualize the family tree of any model on the Hub. It also displays the type of license they use: permissive (green), noncommercial (red), and unknown (gray). It should help people select the right license based on the parent models.

In addition, I hope it can be refined to extract more information about these models: do models from very different branches work better when merged? Can we select them based on the weight difference? There are a lot of questions to explore in this new space. :)

Here's a link to the colab notebook I made: https://colab.research.google.com/drive/1s2eQlolcI1VGgDhqWIANfkfKvcKrMyNr
If you want to know more about model merging or build you own merges, here's the article I wrote about this topic: https://huggingface.co/blog/mlabonne/merge-models

afrideva MaziyarPanahi Jofthomas samusenps flemmingmiguel osanseviero vince62s ucalyptus victor mrfakename luqmanxyz Tradingstrategycourses marcel FiditeNemini Dlbk clem netapy Isotonic satyamt merve dvilasuero Epiculous lunarflu CultriX antiven0m rvpierre mishig Locutusque Citaman Humus macadeliccc ivanfioravanti julien-c acertainbru sugatoray ezzaldeen ahmedelsayed

Dlbk osanseviero mrfakename FiditeNemini clem merve PrimeD Epiculous lunarflu rvpierre Q-bert julien-c kgourgou wsuff nbeerbower

hbzhangvt samusenps osanseviero clem Isotonic merve HuggyMonkey Epiculous lunarflu rvpierre


GPU Poor POV: Low Hanging Fruits

Sometimes we had to work with different language than English (what a surprise!) and it can be problematic, because as you may know many algorithms are mainly developed in English.
I was involved in building RAG in Polish language. At first, we need an proper embeddings for Polish language to feed them into lightweight LLM.
Looking through possible solution I become aware that existing/possible models are not accurate enough, and worked much worse than its 'english equivalent'.
First thing that comes to mind is:
```
 Lets become a mad scientist, download all possible data and train model for months to get the proper one.
```
But there are few cons of this.
- Its computionally heavy
- You are not full time researcher
- you have potential clients who want to use your solution, and they really happy to use it (in optimistic mood).
Here comes the low hanging fruits.
 We developed a easier, workable solution. Instead of training new SOTA, we can use translation module like this one:

https://huggingface.co/Helsinki-NLP/opus-mt-pl-en
translate your knowledge base to english, and use proper embedding model accurately.
I converted existing model using ctranslate2,

```
ct2-transformers-converter --model Helsinki-NLP/opus-mt-pl-en --output_dir opus-mt-pl-en
```
so making an inference  is not heavy (we observe 5 times speedup in compare to original version).

And by indexing knowledge base, we can return answer to LLM  in any language. (Indexes of context found in english language are equal to indexes in native language knowledge base).

Of course there are some tweaks required, we have to validate accuracy of the translation.

It was nice episode, we have our work done, there are people who can use it, so additive value exists.
Have a great day and I wish you more effective deploys! <3

mlabonne Tonic victor Dlbk clem NeuralNovel Ji-Ha

afrideva MaziyarPanahi osanseviero clem saudi-agi


Explaining the 👑 of zero-shot open-vocabulary object detection: OWLv2 🦉
OWLv2 is scaled version of a model called OWL-ViT, so let's take a look at that first. 📝
OWLViT is an open vocabulary object detector, meaning, it can detect objects it didn't explicitly see during the training. 👀
What's cool is that it can take both image and text queries! This is thanks to how the image and text features aren't fused together.

Taking a look at the architecture, the authors firstly do contrastive pre-training of a vision and a text encoder (just like CLIP). They take that model, remove the final pooling layer and attach a lightweight classification and box detection head and fine-tune.
During fine-tuning for object detection, they calculate the loss over bipartite matches. Simply put, loss is calculated over the predicted objects against ground truth objects and the goal is to find a perfect match of these two sets where each object is matched to one object in ground truth.

OWL-ViT is very scalable. You can easily scale most language models or vision-language models because they require no supervision, but this isn't the case for object detection: you still need weak supervision. Moreover, only scaling the encoders creates a bottleneck after a while.

The authors wanted to scale OWL-ViT with more data, so they used OWL-ViT for labelling to train a better detector, "self-train" a new detector on the labels, and fine-tune the model on human-annotated data.

Thanks to this, OWLv2 scaled very well and topped leaderboards on open vocabulary object detection 👑
If you'd like to try it out, I will leave couple of links with apps, notebooks and more in the comments! 🤗

s3nh NeuralNovel joytafty osanseviero not-lain samusenps Skier8402 Erland mlabonne Dlbk taufiqdp Jofthomas clem rreed aidystark johko ayhaidar ceofast


roseking Chrissss Tonic mrfakename NeuralNovel KyonBS radames hayden-donnelly Duskfallcrew DamarJati AMIAUTHOR fumjtea6fg Dlbk afrideva Jofthomas osanseviero umuthopeyildirim FloRul clem den0620 thr3a MarinaraSpaghetti kramp plmsmile mvaloatto fullpwerr abidlabs lunarflu julien-c MoritzLaurer AdinaY mishig TuringsSolutions victor not-lain


Can Large Language Models Explain Themselves? (2401.07927)

🔍 Today's pick in Interpretability & Analysis of LMs: Can Large Language Models Explain Themselves? by @andreasmadsen Sarath Chandar & @sivareddyg

LLMs can provide wrong but convincing explanations for their behavior, and this might lead to ill-placed confidence in their predictions. This study uses self-consistency checks to measure the faithfulness of LLM explanations: if an LLM says a set of words is important for making a prediction, then it should not be able to make the same prediction without these words. Results demonstrate that LLM self-explanations faithfulness of self-explanations cannot be reliably trusted, as they prove to be very task and model dependent, with bigger model generally producing more faithful explanations.

📄 Paper: https://huggingface.co/papers/2401.07927

sivareddyg roseking osanseviero Tonic KnutJaegersberg victor samusenps mlabonne Dlbk clem clefourrier abidlabs


Self-Rewarding Language Models (2401.10020)

Self-Rewarding Language Models

paper page: https://huggingface.co/papers/2401.10020

Fine-tuning Llama 2 70B on three iterations of our approach yields a model that outperforms many existing systems on the AlpacaEval 2.0 leaderboard, including Claude 2, Gemini Pro, and GPT-4 0613

roseking gsarti NeuralNovel Araeynn TheLastBen samusenps Kernel AIIAR eliebak mlabonne bunnycore Norm Dlbk clem orick96


Last week, we released 🤗 Transformers.js v2.14, which added support for SAM (Segment Anything Model).

This means you can now generate high-quality segmentation masks for objects in a scene, directly in your browser! 🤯

Demo (+ source code): https://huggingface.co/spaces/Xenova/segment-anything-web
Model: https://huggingface.co/Xenova/slimsam-77-uniform

But how does this differ from Meta's original demo? 🤔 Didn't that also run in-browser?

Well, in their demo, the image embeddings are computed server-side, then sent to the client for decoding. Trying to do this all client-side would be completely impractical: taking minutes per image! 😵💫

That's where SlimSAM comes to the rescue! SlimSAM is a novel SAM compression method, able to shrink the model over 100x (637M → 5.5M params), while still achieving remarkable results!

The best part? You can get started in a few lines of JavaScript code, thanks to Transformers.js! 🔥

```
// npm i @xenova/transformers
import { SamModel, AutoProcessor, RawImage } from '@xenova/transformers';

// Load model and processor
const model = await SamModel.from_pretrained('Xenova/slimsam-77-uniform');
const processor = await AutoProcessor.from_pretrained('Xenova/slimsam-77-uniform');

// Prepare image and input points
const img_url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/corgi.jpg';
const raw_image = await RawImage.read(img_url);
const input_points = [[[340, 250]]];

// Process inputs and perform mask generation
const inputs = await processor(raw_image, input_points);
const outputs = await model(inputs);

// Post-process masks
const masks = await processor.post_process_masks(outputs.pred_masks, inputs.original_sizes, inputs.reshaped_input_sizes);
console.log(masks);

// Visualize the mask
const image = RawImage.fromTensor(masks[0][0].mul(255));
image.save('mask.png');
```

I can't wait to see what you build with it! 🤗

osanseviero radames rahuldshetty victor brunnolou samusenps afrideva Dlbk clem rreed zuozuo AdamSako omaryshchenko NikitaKras chinmayak cggaurav bmullan doublelotus horizon0514

smalltong02 mlabonne Sylvestre clem rreed Norod78 AkindOf


Posting about a very underrated model that tops paperswithcode across different segmentation benchmarks: OneFormer 👑

OneFormer is a "truly universal" model for semantic, instance and panoptic segmentation tasks ⚔
What makes is truly universal is that it's a single model that is trained only once and can be used across all tasks.
The enabler here is the text conditioning, i.e. the model is given a text query that states task type along with the appropriate input, and using contrastive loss, the model learns the difference between different task types 👇 (see in the image below)

It's also super easy to use with transformers.
```python
from transformers import OneFormerProcessor, OneFormerForUniversalSegmentation

processor = OneFormerProcessor.from_pretrained("shi-labs/oneformer_ade20k_swin_large")
model = OneFormerForUniversalSegmentation.from_pretrained("shi-labs/oneformer_ade20k_swin_large")

# swap the postprocessing and task_inputs for different types of segmentation
semantic_inputs = processor(images=image, task_inputs=["semantic"], return_tensors="pt")
semantic_outputs = model(**semantic_inputs)
predicted_semantic_map = processor.post_process_semantic_segmentation(outputs, target_sizes=[image.size[::-1]])[0]
```
I have drafted a notebook for you to try right away ✨ https://colab.research.google.com/drive/1wfJhoTFqUqcTAYAOUc6TXUubBTmOYaVa?usp=sharing
You can also check out the Space without checking out the code itself 👉 https://huggingface.co/spaces/shi-labs/OneFormer

gsarti rwightman Chuanming Dlbk osanseviero kingabzpro jxtan clem julien-c victor NeuralNovel mlabonne lunarflu

osanseviero samusenps erfanzar clem julien-c Timar Dlbk vztay lunarflu afrideva

osanseviero clem julien-c freinold Dlbk rreed lunarflu noobmldude


Vision Mamba: Efficient Visual Representation Learning with
  Bidirectional State Space Model (2401.09417)

Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

paper page: https://huggingface.co/papers/2401.09417

Recently the state space models (SSMs) with efficient hardware-aware designs, i.e., Mamba, have shown great potential for long sequence modeling. Building efficient and generic vision backbones purely upon SSMs is an appealing direction. However, representing visual data is challenging for SSMs due to the position-sensitivity of visual data and the requirement of global context for visual understanding. In this paper, we show that the reliance of visual representation learning on self-attention is not necessary and propose a new generic vision backbone with bidirectional Mamba blocks (Vim), which marks the image sequences with position embeddings and compresses the visual representation with bidirectional state space models. On ImageNet classification, COCO object detection, and ADE20k semantic segmentation tasks, Vim achieves higher performance compared to well-established vision transformers like DeiT, while also demonstrating significantly improved computation & memory efficiency. For example, Vim is 2.8times faster than DeiT and saves 86.8% GPU memory when performing batch inference to extract features on images with a resolution of 1248times1248. The results demonstrate that Vim is capable of overcoming the computation & memory constraints on performing Transformer-style understanding for high-resolution images and it has great potential to become the next-generation backbone for vision foundation models.

clem Goekdeniz-Guelmez osanseviero merve victor samusenps erfanzar Dlbk TheLastBen mlabonne


Most upvoted papers of 2023 on HF. What do you think are going to be the most prominent research topics in AI for 2024 (also, don't forget to add your papers to the hub this year!).

From: https://huggingface.co/spaces/hysts/daily-papers

pierrci osanseviero vwxyzjn mvaloatto linoyts gsarti victor samusenps julien-c Dlbk yukiarimo mlabonne MaziyarPanahi Jofthomas dvilasuero snarktank hysts mfarajtabar avinash02


Patchscope: A Unifying Framework for Inspecting Hidden Representations
  of Language Models (2401.06102)

💥 Today's pick in Interpretability & Analysis of LMs: 🩺 Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models by @asmadotgh, @codevan, @1wheel, @iislucas & @mega

Patchscopes is a generalized framework for verbalizing information contained in LM representations. This is achieved via a mid-forward patching operation inserting the information into an ad-hoc prompt aimed at eliciting model knowledge. Patchscope instances for vocabulary projection, feature extraction and entity resolution in model representation are show to outperform popular interpretability approaches, often resulting in more robust and expressive information.

🌐 Website: https://pair-code.github.io/interpretability/patchscopes/
📄 Paper: https://huggingface.co/papers/2401.06102

s3nh osanseviero victor codevan iislucas asmadotgh merve Kernel samusenps anjutv clem Dlbk


1) 👩🏾💻 𝐃𝐞𝐜𝐢𝐂𝐨𝐝𝐞𝐫-𝟔𝐁

1) 👩🏾💻 𝐃𝐞𝐜𝐢𝐂𝐨𝐝𝐞𝐫-𝟔𝐁

💻 𝑻𝒓𝒚 𝒊𝒕 𝒐𝒖𝒕:

💻 𝑻𝒓𝒚 𝒊𝒕 𝒐𝒖𝒕:

🃏 𝐌𝐨𝐝𝐞𝐥 𝐂𝐚𝐫𝐝:

🃏 𝐌𝐨𝐝𝐞𝐥 𝐂𝐚𝐫𝐝:

🪧 𝐇𝐮𝐠𝐠𝐢𝐧𝐠𝐅𝐚𝐜𝐞 𝐒𝐩𝐚𝐜𝐞:

🪧 𝐇𝐮𝐠𝐠𝐢𝐧𝐠𝐅𝐚𝐜𝐞 𝐒𝐩𝐚𝐜𝐞:

2) 🎨 𝐃𝐞𝐜𝐢𝐃𝐢𝐟𝐟𝐮𝐬𝐢𝐨𝐧 𝐯𝟐.𝟎

2) 🎨 𝐃𝐞𝐜𝐢𝐃𝐢𝐟𝐟𝐮𝐬𝐢𝐨𝐧 𝐯𝟐.𝟎

💻 𝑻𝒓𝒚 𝒊𝒕 𝒐𝒖𝒕:

💻 𝑻𝒓𝒚 𝒊𝒕 𝒐𝒖𝒕:

🃏 𝐌𝐨𝐝𝐞𝐥 𝐂𝐚𝐫𝐝:

🃏 𝐌𝐨𝐝𝐞𝐥 𝐂𝐚𝐫𝐝:

🪧 𝐇𝐮𝐠𝐠𝐢𝐧𝐠𝐅𝐚𝐜𝐞 𝐒𝐩𝐚𝐜𝐞:

🪧 𝐇𝐮𝐠𝐠𝐢𝐧𝐠𝐅𝐚𝐜𝐞 𝐒𝐩𝐚𝐜𝐞:

✌🏼Two new models dropped today 👇🏽

1) 👩🏾💻 𝐃𝐞𝐜𝐢𝐂𝐨𝐝𝐞𝐫-𝟔𝐁

👉🏽 Supports 𝟖 𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞𝐬: C, C# C++, GO, Rust, Python, Java, and Javascript.

👉🏽 Released under the 𝐀𝐩𝐚𝐜𝐡𝐞 𝟐.𝟎 𝐥𝐢𝐜𝐞𝐧𝐬𝐞

🥊 𝐏𝐮𝐧𝐜𝐡𝐞𝐬 𝐚𝐛𝐨𝐯𝐞 𝐢𝐭𝐬 𝐰𝐞𝐢𝐠𝐡𝐭 𝐜𝐥𝐚𝐬𝐬 𝐨𝐧 𝐇𝐮𝐦𝐚𝐧𝐄𝐯𝐚𝐥: Beats out CodeGen 2.5 7B and StarCoder 7B on most supported languages. Has a 3-point lead over StarCoderBase 15.5B for Python

💻 𝑻𝒓𝒚 𝒊𝒕 𝒐𝒖𝒕:

🃏 𝐌𝐨𝐝𝐞𝐥 𝐂𝐚𝐫𝐝: https://huggingface.co/Deci/DeciCoder-6B

📓 𝐍𝐨𝐭𝐞𝐛𝐨𝐨𝐤: https://colab.research.google.com/drive/1QRbuser0rfUiFmQbesQJLXVtBYZOlKpB

🪧 𝐇𝐮𝐠𝐠𝐢𝐧𝐠𝐅𝐚𝐜𝐞 𝐒𝐩𝐚𝐜𝐞: https://huggingface.co/spaces/Deci/DeciCoder-6B-Demo

2) 🎨 𝐃𝐞𝐜𝐢𝐃𝐢𝐟𝐟𝐮𝐬𝐢𝐨𝐧 𝐯𝟐.𝟎

👉🏽 Produces quality images on par with Stable Diffusion v1.5, but 𝟐.𝟔 𝐭𝐢𝐦𝐞𝐬 𝐟𝐚𝐬𝐭𝐞𝐫 𝐢𝐧 𝟒𝟎% 𝐟𝐞𝐰𝐞𝐫 𝐢𝐭𝐞𝐫𝐚𝐭𝐢𝐨𝐧𝐬

👉🏽  Employs a 𝐬𝐦𝐚𝐥𝐥𝐞𝐫 𝐚𝐧𝐝 𝐟𝐚𝐬𝐭𝐞𝐫 𝐔-𝐍𝐞𝐭 𝐜𝐨𝐦𝐩𝐨𝐧𝐞𝐧𝐭 𝐰𝐡𝐢𝐜𝐡 𝐡𝐚𝐬 𝟖𝟔𝟎 𝐦𝐢𝐥𝐥𝐢𝐨𝐧 𝐩𝐚𝐫𝐚𝐦𝐞𝐭𝐞𝐫𝐬.

👉🏽  Uses an optimized scheduler, 𝐒𝐪𝐮𝐞𝐞𝐳𝐞𝐝𝐃𝐏𝐌++, which 𝐜𝐮𝐭𝐬 𝐝𝐨𝐰𝐧 𝐭𝐡𝐞 𝐧𝐮𝐦𝐛𝐞𝐫 𝐨𝐟 𝐬𝐭𝐞𝐩𝐬 𝐧𝐞𝐞𝐝𝐞𝐝 𝐭𝐨 𝐠𝐞𝐧𝐞𝐫𝐚𝐭𝐞 𝐚 𝐪𝐮𝐚𝐥𝐢𝐭𝐲 𝐢𝐦𝐚𝐠𝐞 𝐟𝐫𝐨𝐦 𝟏𝟔 𝐭𝐨 𝟏𝟎.

👉🏽 Released under the 𝐂𝐫𝐞𝐚𝐭𝐢𝐯𝐞𝐌𝐋 𝐎𝐩𝐞𝐧 𝐑𝐀𝐈𝐋++-𝐌 𝐋𝐢𝐜𝐞𝐧𝐬𝐞.

💻 𝑻𝒓𝒚 𝒊𝒕 𝒐𝒖𝒕:

🃏 𝐌𝐨𝐝𝐞𝐥 𝐂𝐚𝐫𝐝: https://huggingface.co/Deci/DeciDiffusion-v2-0

📓 𝐍𝐨𝐭𝐞𝐛𝐨𝐨𝐤: https://colab.research.google.com/drive/11Ui_KRtK2DkLHLrW0aa11MiDciW4dTuB

🪧 𝐇𝐮𝐠𝐠𝐢𝐧𝐠𝐅𝐚𝐜𝐞 𝐒𝐩𝐚𝐜𝐞: https://huggingface.co/spaces/Deci/DeciDiffusion-v2-0

Help support the projects by liking the model cards and the spaces!

Cheers and happy hacking!

osanseviero merve davanstrien cabralski Quinn-03 kramp itsn78 ailaughface clem alielfilali01 lunarflu Tanvir1337 Dlbk NeuralNovel joytafty VictorSanh den0620 Tonic

osanseviero merve akashicmarga ailaughface clem anjutv ysharma linoyts lunarflu Tanvir1337 Dlbk joytafty samusenps sbrandeis


𝐀𝐧 𝐈𝐝𝐞𝐚 𝐟𝐫𝐨𝐦 𝐭𝐡𝐞 𝐇𝐞𝐚𝐫𝐭

𝐀𝐧 𝐈𝐝𝐞𝐚 𝐟𝐫𝐨𝐦 𝐭𝐡𝐞 𝐇𝐞𝐚𝐫𝐭

𝐄𝐦𝐛𝐫𝐚𝐜𝐞𝐝 𝐛𝐲 𝐇𝐮𝐠𝐠𝐢𝐧𝐠 𝐅𝐚𝐜𝐞: 𝐭𝐡𝐞 𝐈𝐧𝐬𝐢𝐝𝐞 𝐒𝐭𝐨𝐫𝐲 𝐨𝐟 𝐎𝐮𝐫 𝐒𝐭𝐚𝐫𝐭𝐮𝐩’𝐬 𝐀𝐜𝐪𝐮𝐢𝐬𝐢𝐭𝐢𝐨𝐧

In late 2021, our team of five engineers, scattered around the globe, signed the papers to shut down our startup, Gradio. For many founders, this would have been a moment of sadness or even bitter reflection.

But we were celebrating. We were getting acquired by Hugging Face!

We had been working very hard towards this acquisition, but for weeks, the acquisition had been blocked by a single investor. The more we pressed him, the more he buckled down, refusing to sign off on the acquisition. Until, unexpectedly, the investor conceded, allowing us to join Hugging Face.

For the first time since our acquisition, I’m writing down the story in detail, hoping that it may shed some light into the obscure world of startup acquisitions and what decisions founders can make to improve their odds for a successful acquisition.

To understand how we got acquired by Hugging Face, you need to know why we started Gradio.

𝐀𝐧 𝐈𝐝𝐞𝐚 𝐟𝐫𝐨𝐦 𝐭𝐡𝐞 𝐇𝐞𝐚𝐫𝐭

Two years before the acquisition, in early 2019, I was working on a research project at Stanford. It was the third year of my PhD, and my labmates and I had trained a machine learning model that could predict patient biomarkers (such as whether patients had certain diseases or an implanted pacemaker) from an ultrasound image of their heart — as well as a cardiologist.

Naturally, cardiologists were skeptical... read the rest of the story here: https://twitter.com/abidlabs/status/1745533306492588303

osanseviero BrigitteTousi EmilyWitko jeffboudier samusenps harpreetsahota tomaarsen alielfilali01 radames gsarti vinidlidoo pierrci Tonic chansung DopeorNope woolr clem Sd1p ysharma lunarflu susnato rwightman dvilasuero

BrigitteTousi samusenps harpreetsahota tomaarsen gsarti Tonic chansung woolr sbarman25 clem Sd1p kramp lunarflu


🪟32k-context BERT for embedding and RAG on long corpus

Monarch Mixer is a new architecture to enable long context BERT for large corpus and can be fine-tuned for large context retrieval.

Quite interesting and important as BERT is still the most used LLM in production for "old school" tasks like classification, NER, embeddings, but is also a key component for RAG.

Paper: https://arxiv.org/abs/2310.12109
Blog: https://hazyresearch.stanford.edu/blog/2024-01-11-m2-bert-retrieval
GitHub: https://github.com/HazyResearch/m2


Scalable Pre-training of Large Autoregressive Image Models (2401.08541)

Scalable Pre-training of Large Autoregressive Image Models

paper page: https://huggingface.co/papers/2401.08541

paper introduces AIM, a collection of vision models pre-trained with an autoregressive objective. These models are inspired by their textual counterparts, i.e., Large Language Models (LLMs), and exhibit similar scaling properties. Specifically, we highlight two key findings: (1) the performance of the visual features scale with both the model capacity and the quantity of data, (2) the value of the objective function correlates with the performance of the model on downstream tasks. We illustrate the practical implication of these findings by pre-training a 7 billion parameter AIM on 2 billion images, that achieves 84.0% on ImageNet-1k with a frozen trunk. Interestingly, even at this scale, we observe no sign of saturation in performance, suggesting that AIM potentially represents a new frontier for training large-scale vision models. The pre-training of AIM is similar to the pre-training of LLMs, and does not require any image-specific strategy to stabilize the training at scale.

gsarti taufiqdp clem rwightman NeuralNovel


Up to 3x faster LLM generation with no extra resources/requirements - ngram speculation has landed in 🤗 transformers! 🏎💨

All you need to do is to add `prompt_lookup_num_tokens=10` to your `generate` call, and you'll get faster LLMs 🔥

How does it work? 🤔

Start with assisted generation, where a smaller model generates candidate sequences. The net result is a significant speedup if the model agrees with the candidate sequences! However, we do require a smaller model trained similarly 😕

The idea introduced (and implemented) by Apoorv Saxena consists of gathering the candidate sequences from the input text itself. If the latest generated ngram is in the input, use the continuation therein as a candidate! No smaller model is required while still achieving significant speedups 🔥

In fact, the penalty of gathering and testing the candidates is so small that you should use this technique whenever possible!

Here is the code example that produces the outputs shown in the video: https://pastebin.com/bms6XtR4

Have fun 🤗

samusenps Dlbk akashicmarga nickandbro osanseviero abidlabs clem alielfilali01 gblazex DamarJati jaigouk Goekdeniz-Guelmez MaziyarPanahi nbroad abdullah m-ric clefourrier takeraparterer Felladrin

samusenps osanseviero clem alielfilali01 Svngoku takeraparterer

MoritzLaurer osanseviero DRXD1000 manu clem takeraparterer


Real-Time Vehicle Speed Estimation Tutorial 🚗💨💨💨

TL;DR: Watch the tutorial here: https://www.youtube.com/watch?v=uWP6UjDeZvY

Key Steps:
1. Vehicle Detection: Before we jump into speed estimation, we begin by detecting moving vehicles. I demonstrate this using YOLOv8, deployed through the Inference pip package.

2. Tracking with ByteTrack: For effective object tracking, ByteTrack is my tool of choice. It assigns a unique ID to each vehicle, which is essential for accurately monitoring the distance each car travels. This forms the cornerstone of our speed calculation process.

3. Distance Calculation Complexities: Calculating traveled distance can be tricky due to perspective distortion from the camera. A car moving at a constant speed will appear to move a different number of pixels in the image, depending on its distance from the camera.

4. Vehicle Positioning: We can accurately pinpoint each vehicle's position within our monitored area. By representing each vehicle with x and y coordinates in meters, we can compare its current and past positions, paving the way for calculating its speed.

5. We store the position of each car in the last second, calculate the offset, and divide it by the time delta to get the local speed.

- 🔗 tutorial: https://www.youtube.com/watch?v=uWP6UjDeZvY
- 🔗 code: https://github.com/roboflow/supervision/tree/develop/examples/speed_estimation

SkalskiP merve osanseviero julien-c HaohuaLv kramp muhtasham samusenps yeldarb mpfmorawski radames clem jxtan vince62s lynguyenminh victor

osanseviero samusenps radames clem SkalskiP mrfakename


Fine-grained Hallucination Detection and Editing for Language Models (2401.06855)

💥 Today's pick in Interpretability & Analysis of LMs: Fine-grained Hallucination Detection and Editing For Language Models by @abhika-m @akariasai @vidhisha et al.

Authors introduce a new taxonomy for fine-grained annotation of hallucinations in LM generations and propose Factuality Verification with Augmented Knowledge (FAVA), a retrieval-augmented LM fine-tuned to detect and edit hallucinations in LM outputs, outperforming ChatGPT and LLama2 Chat on both detection and editing tasks.

🌐 Website: https://fine-grained-hallucination.github.io
📄 Paper: https://huggingface.co/papers/2401.06855
🚀 Demo: https://huggingface.co/spaces/fava-uw/fava
🤖 Model: https://huggingface.co/fava-uw/fava-model
🔡 Dataset: https://huggingface.co/datasets/fava-uw/fava-data

merve osanseviero samusenps abhika-m yihaopeng d0rj clem julien-c joytafty abdullah dhuynh95 rreed sbrandeis

samusenps abhika-m clem julien-c joytafty rreed


GPU Poor POV: Building a RAG which solves specific task.

Everyone loves benchmarks.
They are great because we have standarized approach,  competitive feeling. But if you are in specific area, trying  to implement some LLM/RAG use case, these benchmarks  cannot exactly reflect on the data that you have to deal with.

I built RAG system on bunch of niche procedures/regulation etc,  which can be finally deployed as an virtual assistant to minimize the effort in searching through a lot of documentations manually.

Tested a lot of different methods/models/pretrains, finetunes and whats interesting is that, final solution which was scored by human feedback is based on relatively low param models, with multitask ability
Something like:

https://huggingface.co/BAAI/llm-embedder

LLMs help summarize the chunk version of knowledge base found, does not require the model with high number of params, because tradeoff between inference time and accuracy has to be made. Some lightweight models have ability to perform certain task based on instructions,  so eg. qwen 7b or mistral 7b (not moe one), realized a task really nicely. And what is more important is that in overall we are able to deploy a RAG system in smaller tasks, in specific area. They can be used by people who need it, give additive value and positive feedback, which IMO is what is all of the building process about.

Have a great day and think about problem which your models have to solve <3

100rab RASMUS victor merve osanseviero tomaarsen samusenps clem Citaman lunarflu Tanvir1337 sethuiyer arpitingle Ji-Ha

merve samusenps akashicmarga clem lunarflu TuringsSolutions Tanvir1337 sbrandeis

victor merve clem lunarflu TuringsSolutions Tanvir1337 s3nh


What is the current SOTA in terms of fast personalized image generation? Most of the techniques that produce great results (which is hard to objectively measure, but subject similarity index being close to 80-90%) take either too much time (full on DreamBooth fine-tuning the base model) or or loose on the auxilary properties (high rank LoRAs).

We have been also testing face embeddings, but even with multiple samples the quality is not anywhere close to what we expect. Even on the techniques that work, high quality (studio-level) pictures seem to be a must so another avenue that I'm curious is whether there is filter/segmentation of the input samples in an automatic way that people have looked in the past?

osanseviero chansung samusenps victor merve clem

samusenps akashicmarga clem badayvedat quantumcomputer


Google's SigLIP is another alternative to openai's CLIP, and it  just got merged to 🤗transformers and it's super easy to use!
To celebrate this, I have created a repository including notebooks and bunch of Spaces on various SigLIP based projects 🥳
Search for art 👉 https://huggingface.co/spaces/merve/draw_to_search_art
Compare SigLIP with CLIP 👉 https://huggingface.co/spaces/merve/compare_clip_siglip

How does SigLIP work?
SigLIP an vision-text pre-training technique based on contrastive learning. It jointly trains an image encoder and text encoder such that the dot product of embeddings are most similar for the appropriate text-image pairs
The image below is taken from CLIP, where this contrastive pre-training takes place with softmax, but SigLIP replaces softmax with sigmoid. 📎

Highlights from the paper on why you should use it ✨
🖼📝 Authors used medium sized B/16 ViT for image encoder and B-sized transformer for text encoder
😍 More performant than CLIP on zero-shot
🗣 Authors trained a multilingual model too!
⚡ Super efficient, sigmoid is enabling up to 1M items per batch, but the authors chose 32k because the performance saturates after that

It's super easy to use thanks to transformers 👇
```python
from transformers import pipeline
from PIL import Image
import requests

# load pipe
image_classifier = pipeline(task="zero-shot-image-classification", model="google/siglip-base-patch16-256-i18n")

# load image
url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)

# inference
outputs = image_classifier(image, candidate_labels=["2 cats", "a plane", "a remote"])
outputs = [{"score": round(output["score"], 4), "label": output["label"] } for output in outputs]
print(outputs)
```
For all the SigLIP notebooks on similarity search and indexing, you can check this [repository](https://github.com/merveenoyan/siglip) out. 🤗

dreamworks2050 victor dvilasuero julien-c osanseviero samusenps Leyo abidlabs TuringsSolutions chansung merve Kernel cdr6934 clem cataluna84

samusenps TuringsSolutions pryanshusharma rwightman chansung linoyts SRDdev akashicmarga yihaopeng clem afrideva cataluna84

nezubn VictorSanh dvilasuero julien-c osanseviero TuringsSolutions chansung clem cataluna84


Is synthetic data the future of AI? 🔥🔥🔥

@HugoLaurencon @Leyo & @VictorSanh are introducing https://huggingface.co/datasets/HuggingFaceM4/WebSight , a multimodal dataset featuring 823,000 pairs of synthetically generated HTML/CSS codes along with screenshots of the corresponding rendered websites to train GPT4-V-like models 🌐💻

While crafting their upcoming foundation vision language model, they faced the challenge of converting website screenshots into usable HTML/CSS codes. Most VLMs suck at this and there was no public dataset available for this specific task, so they decided to create their own.

They prompted existing LLMs to generate 823k HTML/CSS codes of very simple websites. Through supervised fine-tuning of a vision language model on WebSight, they were able to generate the code to reproduce a website component, given a screenshot.

You can explore the dataset here: https://huggingface.co/datasets/HuggingFaceM4/WebSight

What do you think?

Leyo vishaal27 lhoestq AdinaY merve Hotsnow s3nh florentgbelidji HugoLaurencon cabralski ronzyponz mishig HugoDzz victor chansung kramp ameerazam08 wjbmattingly mcpotato taufiqdp osanseviero phanes alielfilali01 duxans aliabd taesiri pierrci lvwerra VictorSanh Triyansha mrfakename dvilasuero Srulikbd julien-c samusenps Chinar PeepDaSlan9 nic221 DEKUBE LouMa Mithilss rohanbalkondekar SaylorTwift peterschmidt85 yihaopeng mkemka radames faust01 lunarflu vztay iblub zeynel rreed

s3nh lhoestq AdinaY BramVanroy merve cabralski davanstrien Pwlot mishig HansKlepitko HugoDzz bstraehle Sylvestre neovalle ameerazam08 wjbmattingly osanseviero aliabd sungkim11 clementperrot cyberandy dvilasuero julien-c samusenps mrdbourke PeepDaSlan9 Frantz103 LouMa Kernel MoritzLaurer yihaopeng lunarflu Kathakali zeynel z11h rreed freddyaboulton

happymanda cabralski mishig ayut HugoDzz clefourrier ameerazam08 osanseviero aliabd dvilasuero julien-c samusenps PeepDaSlan9 DEKUBE rohanbalkondekar MaziyarPanahi yihaopeng lunarflu rreed jeffboudier sbrandeis

taufiqdp cabralski mishig Wauplin holymonkey47 aliabd doxiy dreamworks2050 Srulikbd julien-c lunarflu binga


Happy to announce, brand new, open-source Hugging Face Competitions platform 🚀 Now, create a machine learning competition for your friends, colleagues or the world for FREE* and host it on Hugging Face: the AI community building the future. Creating a competition requires only two steps: `pip install competitions`, then run `competitions create` and create competition by answering a few questions 💥 Checkout the github repo: https://github.com/huggingface/competitions and docs: https://hf.co/docs/competitions

clem merve robmarkcole linoyts kurianbenoy victor alielfilali01 abhishek AdinaY osanseviero Tonic gsarti nezubn VictorSanh julien-c mrdbourke MaziyarPanahi samusenps Smooke d0rj dillfrescott Ammok ecevinoth PinakHolder

happymanda ThetaPhiPsi chansung victor AdinaY osanseviero doxiy julien-c kramp Smooke Solshine dillfrescott Ammok

mc0ps alielfilali01 pacoid osanseviero julien-c TuringsSolutions Smooke mrfakename Solshine dillfrescott IndiraMandalapu

clem dreamworks2050 julien-c dillfrescott


🙋🏻♂hey there folks,

Everyone's🗣talking about microsoft's new e5mistral embeddings model

🤔🤔 but did you actually try it yet ?

Well , now you can, just check it out. it's a new way to serve and create embeddings.

try it hosted on GPUZero : https://huggingface.co/spaces/Tonic/e5
or served on an A10G : https://huggingface.co/spaces/Tonic/e5

you get best results actually building with it though, so use it in your app !

Our demo is coming soon too so let's work together if you want :-)

kramp s3nh clem osanseviero AdinaY abidlabs eddy8383 nickandbro tomaarsen

clem linoyts samusenps sosoai tomaarsen jilp00


Just published a quick community blog post mainly aimed at Art and Design students, but which is also an attempt to nudge AI researchers who would like to better consider benefits from collaboration with designers and artists 😉
Feel free to share your thoughts !

"Breaking Barriers: The Critical Role of Art and Design in Advancing AI Capabilities" 📄 https://huggingface.co/blog/fffiloni/the-critical-role-of-art-and-design-in-advancing-a

—
This short publication follows the results of two AI Workshops that took place at École des Arts Décoratifs - Paris, lead by Etienne Mineur, Vadim Bernard, Martin de Bie, Antoine Pintout & Sylvain Filoni.

victor lbourdois kramp osanseviero julien-c radames tonyassi chansung Tonic pierrci mishig clem merve linoyts AdinaY not-lain lucasjin sbrandeis

smellsworth clem merve not-lain samusenps OmbelineM


💡We recently launched a Discord server for #Dutch #NLP and #LLMs. We have more than 50 users of _very_ varrying backgrounds! 🧙👩🔬🧑🎨🧑🏫🧑💼 We've already had discussions on eval, tokenizers, RAG, data... A bit of everything. Everyone is welcome to work together on Dutch NLP and LLMs!  https://discord.gg/YUUXVZkZJ9

stefan-it lbourdois osanseviero Tonic gsarti samusenps abidlabs mariagrandury yjernite sbrandeis


As the amount of datasets for fine tuning chat models has grown, there's been a plethora of dataset formats emerge. The most popular of these include the formats used by Alpaca, ShareGPT and Open Assistant datasets. The datasets and their formats have also evolved from single-turn conversation to multi-turn. Many of these formats share similarities (and they all have the same goal), but handling the variations in formats across datasets is often a hassle, and source of potential bugs.

Luckily the community seems to be converging on a simple and elegant chat dataset format: a list with each record being an array with each conversation turn being an object with a role (system, assistant or user) and content. Hugging Face uses this input format in the [Templates for Chat Models](https://huggingface.co/docs/transformers/main/en/chat_templating#how-do-i-use-chat-templates) docs:

```python
messages = [
    {
        "role": "system",
        "content": "You are a friendly chatbot who always responds in the style of a pirate",
    },
    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
]
```

Popular datasets like https://huggingface.co/datasets/HuggingFaceH4/no_robots follow this format.

To encourage usage of this format, I propose we give it a name: Hugging Face MessagesList format.

The format is defined as:

- Having at least one `messages` column of type list.
- Each messages record is an array containing one or more message turn objects.
- A message turn must have `role` and `content` keys.
- `role` should be one of `system`, `assistant` or `user`.
- `content` is the text content of the message.

This may be a small thing, but having a common dataset format will reduce wasted time data wrangling and help everyone.

davanstrien osanseviero Weyaxi lhoestq julien-c clem merve alielfilali01 andysalerno dvilasuero samusenps afrideva IFSDrake sbrandeis

victor julien-c Tonic dvilasuero TuringsSolutions madroid davanstrien clefourrier


Want to read curated list of papers by @akhaliq in your mail box?

Thanks to the API provided by Hugging Face, I made a simple GitHub Action based newsletter bot to send out 🤗 Daily Papers. Check out the attached video clip to get a sense of what it is!

Internally, it leverages Gemini API to assign tags for each paper, and all papers are archived by tags and batches. Of course, you can directly go to the papers' pages from your mail box to check out the full paper!

Since everything is automated, GitHub Action and Gemini API are free, and the subscription management is free via Google Groups, this newsletter bot is entirely free. Furthermore, if you wish, you could fork the project for your own newsletter service.

subscription: https://groups.google.com/g/hf-daily-paper-newsletter
project repo: https://github.com/deep-diver/hf-daily-paper-newsletter

In the next step, I will experimentally add auto translation (to Korean) feature for every papers.

chansung Dlbk lbourdois victor osanseviero julien-c Tonic akhaliq pierrci clem mishig merve dashfunnydashdash Moguiy AdinaY abidlabs rwightman veerpareek samusenps ysharma nbroad inmatrix Oliver-Guy

osanseviero julien-c chansung Tonic akhaliq clem linoyts AdinaY mariagrandury samusenps ysharma


🙋🏻♂Hey there folks ,

i wanted to share with you a really cool new organisation called

https://huggingface.co/lowres

In just one week it has gathered almost 150 members !

Check them out if you love anime , SDLX, LORAs and cool datasets.

can we make this one reach 200 members? 🚀

not-lain julien-c clem merve s3nh akashicmarga AdinaY radames samusenps ppaon sbrandeis


🤦🏻♂well, day before yesterday i was so happy about **gpuzero** that i made a bunch of demos : https://huggingface.co/posts/Tonic/802671427380916

- one for YI-200K , but it actually doesnt quite fit on a GPUZero... U_U
- one for SDXL style align, but omg i didnt even realize at the time it wasnt my demo of it (lol)
- one for texify (which works great btw, keep an eye on texify, it's about to blow up... in a couple of months!)

so yeah, i ran back and tried to get my demos working at least for sdxl which i love , but i simply couldnt get the CPU stuff working, or the refactored code working. no wonder i was thinking "wow this is so easy" on @osanseviero 's demo : yeah , it's not my code that's why it works 😅🙏🏻

anyway spent the day unsuccessfully experimenting, but starting tomorrow i'll try to serve some cool and overlooked models so 🤗huggingface appreciators can try them out 🚀

osanseviero hogiahien victor clem julien-c jarvisx17 Solshine


Shocking: 2/3 of LLMs fail at 2K context length

code_your_own_ai makes a great vlog about mostly LLM related AI content.
As I watched the video below, I wondered about current best practices on LLM evaluation. We have benchmarks, we have sota LLMs evaluating LLMs, we have tools evaluating based on human comparison.
Often, I hear, just play with the LLM for 15 mins to form an opinion.
While I think for a specific use case and clear expectations, this could yield signal carrying experiences, I also see that one prompt is used to judge models.
While benchmarks have their weaknesses, and are by themselves not enough to judge model quality, I still think systematic methods that try to reduce various scientifically known errs should be the way forward, even for qualitative estimates.
What do you think? How can we make a public tool for judging models like lmsys/chatbot-arena-leaderboard help to leverage standards known in social science?

https://www.youtube.com/watch?v=mWrivekFZMM

clem osanseviero julien-c pierrci NameeO radames AdinaY Tanvir1337 ucalyptus frontenbrecher


Here is my selection of papers for today (12 Jan)

https://huggingface.co/papers

PALP: Prompt Aligned Personalization of Text-to-Image Models

Object-Centric Diffusion for Efficient Video Editing

TRIPS: Trilinear Point Splatting for Real-Time Radiance Field Rendering

Diffusion Priors for Dynamic View Synthesis from Monocular Videos

Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation

TOFU: A Task of Fictitious Unlearning for LLMs

Patchscope: A Unifying Framework for Inspecting Hidden Representations of Language Models

Secrets of RLHF in Large Language Models Part II: Reward Modeling

LEGO:Language Enhanced Multi-modal Grounding Model

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

Tuning LLMs with Contrastive Alignment Instructions for Machine Translation in Unseen, Low-resource Languages

A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism

Towards Conversational Diagnostic AI

Transformers are Multi-State RNNs

Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training

Distilling Vision-Language Models on Millions of Videos

Efficient LLM inference solution on Intel GPU

TrustLLM: Trustworthiness in Large Language Models

clem osanseviero namtran victor chansung ronzyponz davanstrien Tonic sackoh Dlbk samusenps AdinaY alielfilali01 iarata


Let's play a little game, how would you build the Rabbit R1 with open source tech? Here is my stack:

  - https://huggingface.co/openai/whisper-small for awesome Speech-to-Text with low latency
  - https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1 for an awesome super powerful LLM Brain
  - https://huggingface.co/coqui/XTTS-v2 for a nice and clean voice

Which stack will you personally choose?

kramp osanseviero mwz mohitsha abhishek not-lain clem jahnviatls n4ze3m ManOhMan taufiqdp Sandeepa KnutJaegersberg eek lunarflu jonChengRX0 RalphX1 m-ric timqian rbiswasfc julien-c mrfakename marcsun13 radames johannhartmann Halfshadow Dlbk ucalyptus mishig Moguiy luizgsbraz 10MinuteMan lulzx jesusoctavioas cashion

merve clem yezzer vedalken VictorSanh KnutJaegersberg auditor lunarflu BrigitteTousi RalphX1 loubnabnl julien-c hensam92 mrfakename radames Dlbk mishig clefourrier sosoai SvCy Docfile neiltron 10MinuteMan behlock sbrandeis

clem RalphX1 julien-c pierrci mrfakename


Sharing a super-fast segmentation model today 💨
SlimSAM is pruned-distilled version of SAM model, it's (up-to 8.6x) faster and smaller yet very powerful! ⚡
It has the same architecture as SAM, meaning you can use the 🤗 transformers code for SAM on SlimSAM models ⬇  (yes only 3 lines of code!)
```python
from transformers import pipeline
generator = pipeline(model="nielsr/slimsam-50-uniform", task="mask-generation")
outputs = generator(image)
```
Lastly, I have built an app for you to compare SlimSAM and SAM outputs
https://huggingface.co/spaces/merve/slimsam

osanseviero linoyts abhishek clem lunarflu BrigitteTousi marcsun13 julien-c Pclanglais Dlbk samusenps Tonic sbarman25 pcuenq afrideva sbrandeis


As a GPU poor, I found a nice open source project.

dstackai is the perfect open source project for GPU poor. Simply specify resource requirements (GPU RAM, spot, ...), then will suggest the cheapest options among the popular GPU cloud providers (AWS, GCP, Azure, Lambda Labs, TensorDock, and vast.ai)

Provision VM instances in 3 different use cases. These are the essential for any ML projects.
- Dev: connect provisioned VM instance to your fav IDE (including Jupyter)
- Task: run experiments (training, fine-tuning, ...) via SSH
- Service: run your model in production via HTTPS

dstack is 100% open source, but you need to have your own accounts for each GPU cloud provider, enough GPU quota, configure credentials, etc., all by yourself. Luckily, dstack will announce dstack cloud which let you not worried about all the hassles. The price is almost as same as you directly connect to each cloud with your account.

The attached code snippet shows you how to provision a Mistral-7B model in Text Generation Inference(TGI) on the cheapest VM instance (of having 24GB of VRAM). Then you get the HTTPS connection to it, and play with it as usual with TGI client library as attached in the second code snippet.

If you want to learn more about dstack, check out the official website. Without GPU sponsors, as an individual open source contributor in ML, this kind of project is pretty important.
: https://dstack.ai/

If you are looking for an alternative, there is SkyPilot project as well
: https://github.com/skypilot-org/skypilot

peterschmidt85 chansung osanseviero lysandre merve victor sayakpaul linoyts abhishek abidlabs KnutJaegersberg auditor johannhartmann julien-c radames bitdeep samusenps mishig gblazex juyongjiang greencookies akhaliq Citaman sbrandeis

peterschmidt85 lysandre merve KnutJaegersberg julien-c pierrci samusenps juyongjiang akhaliq


https://huggingface.co/fblgit/UNA-dolphin-2.6-mistral-7b-dpo-laser

RE-Introducing, some of the best SFT model, he legend: DOLPHIN.  This model is very special,  a LASER-UNA model: UNA-dolphin-2.6-mistral-7b-dpo-laser

@fblgit in collaboration with @fernandofernandes and @ehartford

fblgit osanseviero den0620 radames lysandre abhishek victor m-ric julien-c dvilasuero tshrjn robertpiosik pacman2473 AtAndDev


This is my first post! Thanks to @victor for adding me!

lysandre victor sayakpaul abhishek clem julien-c Dlbk


ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of
  Video (2401.05314)

Score Distillation Sampling with Learned Manifold Corrective (2401.05293)

InseRF: Text-Driven Generative Object Insertion in Neural 3D Scenes (2401.05335)

PIXART-δ: Fast and Controllable Image Generation with Latent
  Consistency Models (2401.05252)

URHand: Universal Relightable Hands (2401.05334)

The Impact of Reasoning Step Length on Large Language Models (2401.04925)

Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk (2401.05033)

Here is my selection of papers for today (11 Jan)

https://huggingface.co/papers

ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of Video https://huggingface.co/papers/2401.05314

Score Distillation Sampling with Learned Manifold Corrective https://huggingface.co/papers/2401.05293

InseRF: Text-Driven Generative Object Insertion in Neural 3D Scenes https://huggingface.co/papers/2401.05335

PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Models https://huggingface.co/papers/2401.05252

URHand: Universal Relightable Hands https://huggingface.co/papers/2401.05334

The Impact of Reasoning Step Length on Large Language Models https://huggingface.co/papers/2401.04925

Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk https://huggingface.co/papers/2401.05033


 🙋🏻♂hey there folks , 🌟Tonic here
- just a 🛠builder from 🗼Paris !
Everyone is making something special for their first post , so since i got access to **GPUZero** , well, my first post is about **GPUZero**

### GPUZero is here !

This one's great for builders like me that are often making and serving models to their community.
- demos get popular then fade away
- they retain interest over the next three months as folks have questions

**GPUZero** lets you serve demos to your community over time while optimizing for costs .
Believe it or not it's actually impossible to pay for everything over a whole month if you have even one GPU running at a time.
I'm so excited for this because it lets  me serve a complete stack of specialized models and to build with them too.
- all optimized for efficiency in dollar cost.

check out some demos that are available on GPUZero :
- https://huggingface.co/spaces/Tonic/marker-texify : this one  is the first one i made it's for an image to latex formula model.
-  https://huggingface.co/spaces/Tonic/YI-6B-200k : this one probably actually works better on GPUZero than on a standard A10, but dont take my word for it , try it out 🤗
- https://huggingface.co/spaces/Tonic/style-aligned_sdxl : this one was my greatest technical achievement, check the dates and times on it too, there's a backstory to this one so i'll maybe tell it in another post

osanseviero mlabonne lunarflu hogiahien merve victor julien-c radames bigbeachdaddy pcuenq samusenps Chunte sharad

osanseviero lunarflu merve mrfakename julien-c bigbeachdaddy pcuenq


I finished my model merging experiment day.🤗I would love your thoughts on this.

What did I do? I merged Mistral Instruct 0.1 and 0.2 models using different merging techniques:
- SLERP: linear interpolation (most popular method)
- MoE: replace some forward layers with MoE layers; using a random gate for now
- Frankenmerge: also known as passthrough, but that isn't very cool. It concatenates some specified layers ending in different numbers of params. In my case, I went from 7B to 9B.

Note: merging is not building an ensemble of models. You can read more about merging techniques at https://huggingface.co/blog/mlabonne/merge-models

Results
I built the 3 models using mergekit (running in an HF Space) - took less than an hour to do the three) https://huggingface.co/collections/osanseviero/mistral-instruct-merges-659ebf35ca0781acdb86bb0a

I'm doing a quick check with the OpenLLM Leaderboard.
🚨The OpenLLM Leaderboard is more suitable for pre-trained models than instruct models, but I still thought it would be interesting to look at the insights🚨

You can look at the attached image. Some interesting things
- All three models performed somewhere between 0.1 and 0.2 - congrats to the 140 people who got it right in https://twitter.com/osanseviero/status/1745071548866736171
- Frankenmerge terribly sucked with GSM8K. It seems that adding some Mistral 0.1 layers actually degraded the performance a lot  - this is worse than even 0.1!
- Otherwise, frankenmerge was decent across HellaSwag, MMLU, and specially TruthfulQA
- MoE is using random gating, so I expected something right in between 0.1 and 0.2, which was the case

What do I do with this?
Not sure tbh! I think doing proper MT bench evals would be nice. I also think all of us should give a nice GH star to mergekit because it's awesome. I would love to have the time to do end-to-end ablation studies, but cool new things are coming up. Let me know if you have any thoughts in the results

davanstrien merve mlabonne mrm8488 abhibisht89 gblazex eitanturok detakarang dvilasuero victor julien-c kgourgou thomwolf mwitiderrick not-lain pcuenq marcsun13 clem chansung radames lysandre KnutJaegersberg toshas Mohammadreza m-ric ernestp56 wenqiglantz bhadresh-savani Dlbk nouamanetazi samusenps Csplk Warung jiandong chrissarmstrong sbrandeis Haleshot

davanstrien pprp dvilasuero julien-c not-lain mrfakename clem chansung lysandre KnutJaegersberg DrishtiSharma nouamanetazi macadeliccc

mrm8488 julien-c not-lain clem chansung Haleshot


Last month was great for faster/smaller segmentation models, and I wanted to dedicate my first post to compile the recently released SAM variants! 🤗
📚 All models and their demos can be found in this collection 👉🏼 https://huggingface.co/collections/merve/segment-anything-model-6585835fc76915aa14e2bcbd
The ideas behind them are mostly about making heavy image encoder lighter either through distillation or changing the pre-training. 💡
⚡MobileSAM: It decouples the heavy image encoder of SAM and distills it into a TinyViT to make SAM smaller. The architecture is same except for the encoder.
⚡TinySAM: It distills the whole model with online hard prompt sampling. The authors also quantized it and released Q-TinySAM.
⚡ EfficientSAM: This model combines masked image pre-training for training lightweight image encoders (like ViTMAE, learns to reconstruct the images) and mask decoder.
⚡ FastSAM: It's a CNN-based model where the problem is modeled as segments generation. The inference takes place as everything is segmented at once and then you can prompt with boxes or points or text (and this is how it is similar to SAM). So the architecture is nowhere similar to original SAM itself.
✨ [NEW] SlimSAM: It's a pruned-distilled version of pre-trained SAM. The architecture is same so @nielsr recently converted the weights and you can use it with the same API you use with SAM models. You can find the available checkpoints in the collection.
I hope you liked it!

osanseviero lopezjhonf NemesisAlm amyeroberts julien-c mattmdjaga KnutJaegersberg bisnotforbella AdinaY samusenps abidlabs

Norod78 julien-c KnutJaegersberg mlabonne AdinaY sbrandeis


🔥 Less is more for DPO, high quality matters!

📢 Dropping our first open dataset and LLM of the year:

💾Meet distilabel Orca Pairs DPO, an improved version of the now famous dataset from Intel:

https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs

🏛 And a new OpenHermes fine-tune outperforming baselines with 54% less DPO pairs:

https://huggingface.co/argilla/distilabeled-Hermes-2.5-Mistral-7B

You can use this new dataset for your DPO tuning, just like this:

```
from datasets import load_dataset

# Instead of this:
# dataset = load_dataset("Intel/orca_dpo_pairs", split="train")

# use this:
dataset = load_dataset("argilla/distilabel-intel-orca-dpo-pairs", split="train")

dataset = dataset.filter(
    lambda r:
        r["status"] != "tie" and
        r["chosen_score"] >= 8 and
        not r["in_gsm8k_train"]
)
```
This will reduce the size of the original by 54% while giving you better quality preferences!

What should we build next?

davanstrien osanseviero merve radames tomaarsen mlabonne alielfilali01 clem linoyts KnutJaegersberg Forbu14 lysandre d0rj mammour VictorSanh gblazex sethuiyer eliebak sbrandeis

davanstrien osanseviero merve radames tomaarsen BramVanroy clem KnutJaegersberg lixin67


Finally found my go-to hat for 2024 😎

Thanks to https://huggingface.co/fal-ai

dvilasuero burkaygur radames merve pierrci not-lain clem clefourrier samusenps MarinaraSpaghetti fedyanin dillfrescott roseking sbrandeis

lysandre merve clem roseking dillfrescott


Here is my selection of papers for today (10 Jan)

https://huggingface.co/papers

Jump Cut Smoothing for Talking Heads

FADI-AEC: Fast Score Based Diffusion Model Guided by Far-end Signal for Acoustic Echo Cancellation

Masked Audio Generation using a Single Non-Autoregressive Transformer

Let's Go Shopping (LGS) -- Web-Scale Image-Text Dataset for Visual Concept Understanding

Narrowing the Knowledge Evaluation Gap: Open-Domain Question Answering with Multi-Granularity Answers

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding

MagicVideo-V2: Multi-Stage High-Aesthetic Video Generation

not-lain pierrci julien-c abidlabs davanstrien osanseviero

not-lain julien-c abidlabs davanstrien alielfilali01


hello world!
we're starting a new recurring event/club where we read and implement cool  ai papers on skunkworks discord. first paper we chose is self-play as there are a lot of opportunities to expand on this framework, here's the link for the event: https://discord.gg/eAgBr7Fy?event=1194392774905172030

im plannin my next post to be a technical deepdive of PCN and ProspectiveConfiguration algo as ive been spending the last few days getting a good grasp at this promising alternative to BP, stay tuned.

osanseviero Tonic victor pierrci pcuenq merve samusenps sbrandeis


Here is my selection of papers for today (9 Jan)

https://huggingface.co/papers

AGG: Amortized Generative 3D Gaussians for Single Image to 3D

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

DiarizationLM: Speaker Diarization Post-Processing with Large Language Models

TeleChat Technical Report

Soaring from 4K to 400K: Extending LLM's Context with Activation Beacon

AST-T5: Structure-Aware Pretraining for Code Generation and Understanding

Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach

Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM

GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation

CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution

Mixtral of Experts

mwz alckasoc Tonic victor merve davanstrien IlyasMoutawwakil lixin67 aust-t


Full fine-tuning of Microsoft's Phi2 on a single 4090 is now supported in axolotl. Thanks to @abacaj and @vikhyatk for their help with gradient checkpointing and flash attention fixes.

alpaca finetune: https://huggingface.co/openaccess-ai-collective/phi2-alpaca
wandb: https://wandb.ai/oaaic/phi2/runs/00pc4ugb?workspace=user-wing-lian
merged PR: https://github.com/OpenAccess-AI-Collective/axolotl/pull/1058

osanseviero Citaman victor kramp Tonic marcsun13 merve ybelkada Zmu samusenps mrfakename Chunte JaiSurya radames

osanseviero Tonic merve ybelkada Chunte radames SicariusSicariiStuff sbrandeis


Here is my selection of papers for today (8 Jan)

https://huggingface.co/papers

DocGraphLM: Documental Graph Language Model for Information Extraction

Denoising Vision Transformers

Progressive Knowledge Distillation Of Stable Diffusion XL Using Layer Level Loss

Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively

Pheme: Efficient and Conversational Speech Generation

DeepSeek LLM: Scaling Open-Source Language Models with Longtermism

Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache


QuIP# ecosystem is growing :)

I've seen a quip# 2 bit Qwen-72b-Chat model today on the hub that shows there is support for vLLM inference.
This will speed up inference and make high performing 2 bit models more practical. I'm considering quipping MoMo now, as I can only use brief context window of Qwen-72b on my system otherwise, even with bnb double quantization.

https://huggingface.co/keyfan/Qwen-72B-Chat-2bit

Also notice the easier to use Quip# for all library :)

https://github.com/chu-tianxiang/QuIP-for-all

Felladrin osanseviero victor ybelkada reach-vb merve samusenps mexicanamerican

osanseviero ybelkada reach-vb Tonic kramp merve abidlabs

osanseviero ybelkada reach-vb merve samusenps sbrandeis


👋 Hi there!

This is my very first post.

I'll use  it to share some old news: a math preference dataset for DPO!

I created this dataset some time ago while we were developing distilabel (https://github.com/argilla-io/distilabel).

Some days ago we found out people are actually using it! So I'll use this post to explain how I built it in case it's useful for the community.

1. I used distilabel's SelfInstruct-inspired task to generate instructions about different math topics. I curated the instructions with Argilla (on Spaces!).
2. Then I used a distilabel Pipeline to build a preference dataset using gpt3.5 as generator and gpt4 as labeller. If I recall correctly I used our JudgeLM implementation (see https://distilabel.argilla.io/latest/technical-reference/tasks/#judgelmtask)

(see the screenshot with the dataset in the Argilla UI)

3. Then I just binarized into chosen, rejected pairs and voilà:

https://huggingface.co/datasets/argilla/distilabel-math-preference-dpo

The funny thing is that I used this to do a second DPO run over Notus-7B. I hoped to see an improvement on math/reasoning skills but it actually improved in STEM and Humanities and did worse on Math 🤣 .

In conclusion, this dataset was only a quick experiement. I'm happy to see the community found it useful. Data for DPO and fine-tuning are still a mystery, let's unveil these mysteries in 2024 together!

Follow me for the most exciting datasets for LLMs (and maybe some great, small, efficient models). I plan to announce all Argilla open-source work here!

pacoid sparkycollier osanseviero gabrielmbmb mishig severo sugatoray davanstrien mlabonne qJakc MoritzLaurer KnutJaegersberg radames victor clem julien-c merve tmnam20 Erland DarkLord7771 abidlabs Masa-Erland

pcuenq osanseviero gabrielmbmb mishig plaguss davanstrien pierrci qJakc thomwolf Manel-Hik victor radames mariagrandury clem julien-c merve sbrandeis

qJakc Tonic mariagrandury julien-c davanstrien merve


Train ANYTHING on Hugging Face Spaces hardware using AutoTrain SpaceRunner: https://hf.co/blog/stefan-it/autotrain-flair-mobie 💥

mrfakename abhishek smangrul nouamanetazi dvilasuero thomwolf clem sbrandeis


Here is my selection of papers for today (5 Jan)

https://huggingface.co/papers

Learning the 3D Fauna of the Web

Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation

Improving Diffusion-Based Image Synthesis with Context Prediction

LLaMA Pro: Progressive LLaMA with Block Expansion

LLaVA-φ: Efficient Multi-Modal Assistant with Small Language Model

Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers

FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understanding

TinyLlama: An Open-Source Small Language Model

What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs

LLM Augmented LLMs: Expanding Capabilities through Composition

ODIN: A Single Model for 2D and 3D Perception

Instruct-Imagen: Image Generation with Multi-modal Instruction

ICE-GRT: Instruction Context Enhancement by Generative Reinforcement based Transformers

Understanding LLMs: A Comprehensive Overview from Training to Inference


DreamTalk: When Expressive Talking Head Generation Meets Diffusion
  Probabilistic Models (2312.09767)

I just published a Gradio demo for AliBaba's DreamTalk 🤗

Try it now: https://huggingface.co/spaces/fffiloni/dreamtalk
Paper: https://huggingface.co/papers/2312.09767
—
DreamTalk is a diffusion-based audio-driven expressive talking head generation framework that can produce high-quality talking head videos across diverse speaking styles. DreamTalk exhibits robust performance with a diverse array of inputs, including songs, speech in multiple languages, noisy audio, and out-of-domain portraits.

cnmoro gz888 reach-vb osanseviero abidlabs radames nouamanetazi victor clem julien-c merve lunarflu IbrahimSaidi sbrandeis

santiviquez osanseviero alvarobartt reach-vb clem julien-c merve lunarflu akhaliq pierrci sbrandeis


Here is my selection of papers for today (4 Jan)

https://huggingface.co/papers

Efficient Hybrid Zoom using Camera Fusion on Mobile Phones

Incremental FastPitch: Chunk-based High Quality Text to Speech

CoMoSVC: Consistency Model-based Singing Voice Conversion

SIGNeRF: Scene Integrated Generation for Neural Radiance Fields

From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations

aMUSEd: An Open MUSE Reproduction

Image Sculpting: Precise Object Editing with 3D Geometry Control

A Vision Check-up for Language Models

Multilingual Instruction Tuning With Just a Pinch of Multilinguality

WordArt Designer API: User-Driven Artistic Typography Synthesis with Large Language Models on ModelScope

Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditions

GPT-4V(ision) is a Generalist Web Agent, if Grounded

mwz davanstrien nouamanetazi mlninad m-ric victor merve


Currently attempting to hack EvoDiff to generate binders for target proteins with some interesting results. The generated binders tend to change conformation, sometimes drastically, when bound to the target proteins compared to their unbound states. Below is the target protein with an IDR linker, the generated binder, and the binder bound to the target protein with the IDR linker structure as predicted by ESMFold. Notice how the binder goes from being a solid alpha-helix, to being beta-sheets (in orange). That's quite a change in tertiary structure!

osanseviero reach-vb victor nouamanetazi clem mlninad Jemseen mishig merve radames thomwolf samusenps vicgalle ScienceStanley sbrandeis

clefourrier alielfilali01 thomwolf samusenps ScienceStanley assiduous006


Here is my selection of papers for today (3 Jan) on Hugging Face

paper pages: https://huggingface.co/papers

En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data

Boundary Attention: Learning to Find Faint Boundaries at Any Resolution

Taming Mode Collapse in Score Distillation for Text-to-3D Generation

LLaMA Beyond English: An Empirical Study on Language Capability Transfer

Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models

A Comprehensive Study of Knowledge Editing for Large Language Models

Q-Refine: A Perceptual Quality Refiner for AI-Generated Image

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

DocLLM: A layout-aware generative language model for multimodal document understanding

VideoDrafter: Content-Consistent Multi-Scene Video Generation with LLM

TrailBlazer: Trajectory Control for Diffusion-Based Video Generation

Tonic Chunte osanseviero nouamanetazi mlninad merve diwank


Microsoft: Improving Text Embeddings with Large Language Models

- uses an LLM instead of complex pipelines to create the  training data
- directly generates data for numerous text embedding tasks
- fine tunes standard models with contrastative loss achieving great performance
- critical thought: isn't this kinda benchmark hacking? If the benchmarks are so encompassing that they capture the complete idea of embedding, it's maybe a good idea, but often it is oversimplifying, I find.

Feel free to share your thoughts, even if they like mine don't beat the benchmarks ;P

https://arxiv.org/abs/2401.00368

osanseviero pcuenq julien-c alvarobartt nouamanetazi mlninad dvilasuero Praise2112 merve sbrandeis

osanseviero julien-c davanstrien dvilasuero


Here is my selection of papers for today (2 Jan) on Hugging Face

https://huggingface.co/papers

SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity

Boosting Large Language Model for Speech Synthesis: An Empirical Study

COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training

Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws

GeoGalactica: A Scientific Large Language Model in Geoscience

Improving Text Embeddings with Large Language Models

Unicron: Economizing Self-Healing LLM Training at Scale

nouamanetazi merve clem samusenps AdinaY Tonic


✅ Ever wondered how to measure transparency in model development?

My last open-source contribution for 2023 is s Space that allows you to self-assess the transparency of your model based on the 100 indicators of the Foundation Model Transparency Index (FMTI).

The original study evaluated the developers of 10 top LLMs. Curious about how yours measures up? 👀

https://huggingface.co/spaces/mariagrandury/fmti-transparency-self-assessment

Let's commit to a 2024 with greater transparency in the AI ecosystem! 🚀

lbourdois victor osanseviero abhishek Wauplin julien-c davanstrien nouamanetazi dvilasuero merve MaziyarPanahi clem afrideva sbrandeis

osanseviero victor julien-c dvilasuero merve clem


𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

⚡ 𝐓𝐡𝐢𝐬 𝐦𝐨𝐧𝐭𝐡'𝐬 𝐦𝐨𝐬𝐭 𝐢𝐦𝐩𝐨𝐫𝐭𝐚𝐧𝐭 𝐛𝐫𝐞𝐚𝐤𝐭𝐡𝐫𝐨𝐮𝐠𝐡: 𝐃𝐢𝐟𝐟𝐞𝐫𝐞𝐧𝐭𝐢𝐚𝐥 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫 𝐯𝐚𝐬𝐭𝐥𝐲 𝐢𝐦𝐩𝐫𝐨𝐯𝐞𝐬 𝐚𝐭𝐭𝐞𝐧𝐭𝐢𝐨𝐧 ⇒ 𝐛𝐞𝐭𝐭𝐞𝐫 𝐫𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐚𝐧𝐝 𝐟𝐞𝐰𝐞𝐫 𝐡𝐚𝐥𝐥𝐮𝐜𝐢𝐧𝐚𝐭𝐢𝐨𝐧𝐬!

Thought that self-attention could not be improved anymore?

Microsoft researchers have dropped a novel "differential attention" mechanism that amplifies focus on relevant context while canceling out noise. It sounds like a free lunch, but it does really seem to vastly improve LLM performance!

𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

🧠 Differential attention computes the difference between two separate softmax attention maps, canceling out noise and promoting sparse attention patterns

🔥 DIFF Transformer outperforms standard Transformers while using 35-40% fewer parameters or training tokens

📏 Scales well to long contexts up to 64K tokens, leveraging increasing context length more effectively

🔎 Dramatically improves key information retrieval, enhancing in-context learning, and possibly reducing risk of hallucinations 🤯

🔢 Reduces activation outliers, potentially enabling lower-bit quantization without performance drop!

⚙ Can be directly implemented using existing FlashAttention kernels

This new architecture could lead much more capable LLMs, with vastly improved strengths in long-context understanding and factual accuracy.

But they didn’t release weights on the Hub: let’s wait for the community to train the first open-weights DiffTransformer! 🚀

Read their paper 👉 https://huggingface.co/papers/2410.05258


Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7 Fully Tested as Well, Not Only for Realism But Also for Stylization - 15 vs 256 images having datasets compared as well (expressions / emotions tested too) - Used Kohya GUI for training

Full files and article : https://www.patreon.com/posts/112099700

Download images in full resolution to see prompts and model names

All trainings are done with Kohya GUI, perfectly can be done locally on Windows, and all trainings were 1024x1024 pixels

Fine Tuning / DreamBooth works as low as 6 GB GPUs (0 quality degrade totally same as 48 GB config)

Best quality of LoRA requires 48 GB GPUs , 24 GB also works really good and minimum 8 GB GPU is necessary for LoRA (lots of quality degrade)

Full size grids are also shared for the followings: https://www.patreon.com/posts/112099700

Additionally, I have shared full training entire logs that you can see each checkpoint took time. I have shared best checkpoints, their step count and took time according to being either LoRA, Fine Tuning or Batch size 1 or 7 or 15 images or 256 images, so a very detailed article regarding completed.

Check the images to see all shared files in the post.

Furthermore, a very very detailed analysis having article written and all latest DreamBooth / Fine Tuning configs and LoRA configs are shared with Kohya GUI installers for both Windows, Runpod and Massed Compute.

Moreover, I have shared new 28 realism and 37 stylization testing prompts.

Current tutorials are as below:

Windows requirements CUDA, Python, cuDNN, and such : https://youtu.be/DrhUHnYfwC0

How to use SwarmUI : https://youtu.be/HKX8_F1Er_w

How to use FLUX on SwarmUI : https://youtu.be/bupRePUOA18

How to use Kohya GUI for FLUX training : https://youtu.be/nySGu12Y05k

How to use Kohya GUI for FLUX training on Cloud (RunPod and Massed Compute) : https://youtu.be/-uhL2nW7Ddw


we have a leaderboard for video LLMs, and most of the top models are open ones! https://huggingface.co/spaces/opencompass/openvlm_video_leaderboard 👑👏
we are so back 🔥


Baichuan-Omni Technical Report (2410.08565)

为什么我会在作者列表上，是huggingface人名匹配的吗？

为什么我会在作者列表上，是huggingface人名匹配的吗？


I have finally completed a working full Azure and Microsoft MS Graph API implementation which can use all the interesting MS AI features in M365 products to manage CRUD patterns for the graph features across products.

This app shows initial implementation of security, authentication, scopes, and access to Outlook, Calendar, Tasks, Onedrive and other apps for CRUD pattern as AI agent service skills to integrate with your AI workflow.

Below are initial screens showing integration:

URL: https://huggingface.co/spaces/awacke1/MSGraphAPI
Discussion: https://huggingface.co/spaces/awacke1/MSGraphAPI/discussions/5

Best of AI on @Azure and @Microsoft on @HuggingFace :
https://huggingface.co/microsoft
https://www.microsoft.com/en-us/research/
---
Aaron


Multimodal Ichigo Llama 3.1 - Real Time Voice AI 🔥

> WhisperSpeech X Llama 3.1 8B
> Trained on 50K hours of speech (7 languages)
> Continually trained on 45hrs 10x A1000s
> MLS -> WhisperVQ tokens -> Llama 3.1
> Instruction tuned on 1.89M samples
> 70% speech, 20% transcription, 10% text
> Apache 2.0 licensed ⚡

Architecture:
> WhisperSpeech/ VQ for Semantic Tokens
> Llama 3.1 8B Instruct for Text backbone
> Early fusion (Chameleon)

I'm super bullish on HomeBrew/ Jan and early fusion, audio and text, multimodal models!

(P.S. Play with the demo on Hugging Face: https://huggingface.co/spaces/jan-hq/Ichigo-llama3.1-s-instruct)

vigos YaTharThShaRma999 chenduo Yadukrishnan edwixx


Just started going through the latest "State of AI Report 2024", and I cannot get over the predictions!

The report predicts major developments in AI over the next 12 months, including a $10B+ investment from a sovereign state into a large US AI lab, triggering national security scrutiny, and a viral app created by someone without coding skills.

It forecasts changes in data collection practices due to frontier labs facing trials, softer-than-expected EU AI Act implementations, and the rise of an open-source alternative to OpenAI GPT-4 outperforming in benchmarks.

NVIDIA’s dominance will remain largely unchallenged, investment in humanoid robots will decline, Apple’s on-device AI research will gain momentum, and a research paper by an AI scientist will be accepted at a major conference.

Lastly, a GenAI-based video game is expected to achieve breakout success.

Yet to go through all 200+ pages... will post summarized thoughts later.


she assert on my device until i give up AHAHEGHFDGHJHASUFSHD


MiniSearch is celebrating its 1st birthday! 🎉

Exactly one year ago, I shared the initial version of this side-project on Hugging Face. Since then, there have been numerous changes under the hood. Nowadays it uses [Web-LLM](https://github.com/mlc-ai/web-llm), [Wllama](https://github.com/ngxson/wllama) and [SearXNG](https://github.com/searxng/searxng). I use it daily as my default search engine and have done my best to make it useful. I hope it's interesting for you too!

HF Space: https://huggingface.co/spaces/Felladrin/MiniSearch
Embeddable URL: https://felladrin-minisearch.hf.space

Locutusque dillfrescott shtefcs John6666 Zmu Nymbo DeathGodlike KatyaKnw


🎓 Introducing Svitppt.com.ua Presentations Dataset - https://huggingface.co/datasets/nyuuzyou/svitppt

Dataset highlights:
- 18,001 presentations from svitppt.com.ua, a platform for storing and viewing presentations for Ukrainian school students
- Primarily in Ukrainian, with some Russian and English content
- Each entry includes: URL, title, download URL, filepath, and extracted text content (where available)
- Contains original PPT/PPTX files in addition to metadata
- Data covers a wide range of educational topics and presentation materials
- Dedicated to the public domain under Creative Commons Zero (CC0) license

The dataset can be used for analyzing educational presentation content in Ukrainian and other languages, text classification tasks, and information retrieval systems. It's particularly valuable for examining trends in educational presentation materials and sharing practices in the Ukrainian-speaking student community. The inclusion of original files allows for in-depth analysis of presentation formats and structures commonly used in Ukrainian educational settings.


Here is my selection of papers for today

https://huggingface.co/papers

Compact Neural Graphics Primitives with Learned Hash Probing

Restoration by Generation with Constrained Priors
SSR-Encoder: Encoding Selective Subject

Representation for Subject-Driven Generation

Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object Structure via HyperNetworks

InsActor: Instruction-driven Physics-based Characters
Unsupervised Universal Image Segmentation

Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis

DreamGaussian4D: Generative 4D Gaussian Splatting

City-on-Web: Real-time Neural Rendering of Large-scale Scenes on the Web

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision

DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaption by Combining 3D GANs and Diffusion Priors

Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action

Prompt Expansion for Adaptive Text-to-Image Generation

PanGu-Draw

I2V-Adapter: A General Image-to-Video Adapter for Video Diffusion Models

The LLM Surgeon

MathPile: A Billion-Token-Scale Pretraining Corpus for Math

MobileVLM : A Fast, Reproducible and Strong Vision Language Assistant for Mobile Devices

TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones

osanseviero Chunte victor teowu nouamanetazi aaditya sbrandeis


💬 Notux 8x7b has already its own Chat UI running on 🤗 Spaces! Feel free to give it a try and chat with Notux, and let us know how it goes.

https://huggingface.co/spaces/argilla/notux-chat-ui

Kudos to @gabrielmbmb!

gabrielmbmb osanseviero mishig Chunte ceyda sayakpaul thomwolf nouamanetazi dvilasuero pcuenq NovoCode

victor osanseviero Chunte dvilasuero pcuenq NovoCode sbrandeis


Repurposing Diffusion-Based Image Generators for Monocular Depth
  Estimation (2312.02145)

Introducing Marigold 🌼 - a universal monocular depth estimator, delivering incredibly sharp predictions in the wild! Based on Stable Diffusion, it is trained with synthetic depth data only and excels in zero-shot adaptation to real-world imagery. Check it out:

🤗 Hugging Face Space: https://huggingface.co/spaces/toshas/marigold
🤗 Hugging Face Model: https://huggingface.co/Bingxin/Marigold
🤗 Hugging Face Paper: https://huggingface.co/papers/2312.02145
🌐 Website: https://marigoldmonodepth.github.io
👾 Code: https://github.com/prs-eth/marigold
👾 Code: `pip install diffusers` (check comments to this post for details!)
📄 Paper: https://arxiv.org/abs/2312.02145

Brought to you by the fantastic team from the Photogrammetry and Remote Sensing group of ETH Zurich: Bingxin Ke (@Bingxin), Anton Obukhov (@toshas), Shengyu Huang, Nando Metzger (@nandometzger), Rodrigo Caye Daudt, and Konrad Schindler.

osanseviero akhaliq nandometzger victor toshas pwais Chunte radames sayakpaul Wauplin Tonic nouamanetazi julien-c AdinaY clem Citaman abidlabs samusenps ixaxaar sbrandeis

osanseviero akhaliq nandometzger toshas Chunte radames Wauplin julien-c clem sbrandeis

osanseviero akhaliq toshas Chunte julien-c clem Tonic sbrandeis


Do you use Python to visualize data? Wouldn't it be nice if an AI chatbot can help you write Python code and improve your visualization automatically?

Check out our new blog post on how to build an AI chatbot to run code and tweak plots: https://huggingface.co/blog/sophiamyang/tweak-mpl-chat

sophiamyang philippjfr osanseviero victor kramp ahuang11 Chunte MarcSkovMadsen droumis pcuenq nouamanetazi julien-c merve sbrandeis YAYA20

Chunte MarcSkovMadsen droumis dleybel julien-c samusenps sourceoftruthdata sbrandeis YAYA20

Chunte MarcSkovMadsen droumis julien-c sbrandeis YAYA20


kramp Citaman ariG23498 mwitiderrick abhishek akashicmarga rishiraj mergisi pwais Chunte gpt3eth VictorSanh nouamanetazi julien-c lunarflu BrigitteTousi radames JackCloudman yuchenlin clem antiven0m santiviquez


some mandatory hello world as first post 🤗

```
from tokenizers import Tokenizer
tokenizer = Tokenizer.from_pretrained("bert-base-uncased")
tokenizer.encode("Hello world~ Hello 2024").tokens
```

chansung PSegs osanseviero kramp abhishek mergisi victor Chunte radames Tonic nouamanetazi mishig julien-c samusenps benvdv rreed Sengil

mariagrandury Chunte radames pcuenq mishig julien-c clefourrier gabrielchua rreed


Here is my selection of papers for today (27 Dec) on Hugging Face daily papers newsletter

daily pagers feed: https://huggingface.co/papers

UniRef++: Segment Every Reference Object in Spatial and Temporal Spaceshttps://huggingface.co/papers/2312.15715

LangSplat: 3D Language Gaussian Splattinghttps://huggingface.co/papers/2312.16084

Human101: Training 100+FPS Human Gaussians in 100s from 1 Viewhttps://huggingface.co/papers/2312.15258

Audiobox: Unified Audio Generation with Natural Language Promptshttps://huggingface.co/papers/2312.15821

HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3Dhttps://huggingface.co/papers/2312.15980

One-dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationshttps://huggingface.co/papers/2312.16145

Make-A-Character: High Quality Text-to-3D Character Generation within Minuteshttps://huggingface.co/papers/2312.15430

A Recipe for Scaling up Text-to-Video Generation with Text-free Videoshttps://huggingface.co/papers/2312.15770

Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4https://huggingface.co/papers/2312.16171

SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalinghttps://huggingface.co/papers/2312.15166

Supervised Knowledge Makes Large Language Models Better In-context Learnershttps://huggingface.co/papers/2312.15918

Gemini vs GPT-4V: A Preliminary Comparison and Combination of Vision-Language Models Through Qualitative Caseshttps://huggingface.co/papers/2312.15011

clefourrier osanseviero gz888 Decombas 0x90e ceyda wchai ClayFace mariagrandury Chunte victor nouamanetazi dvilasuero sbrandeis

victor osanseviero radames gz888 Decombas wchai rbiswasfc mwitiderrick abhishek Chunte Tonic dvilasuero


💨 Notux 8x7b was just released!

From Argilla, we recently fine-tuned Mixtral 8x7b Instruct from Mistral AI using DPO, and a binarized and curated version of UltraFeedback, to find out it outperforms every other MoE-based model on the Hub.

- https://huggingface.co/argilla/notux-8x7b-v1
- https://huggingface.co/datasets/argilla/ultrafeedback-binarized-preferences-cleaned

osanseviero gabrielmbmb plaguss 1littlecoder dvilasuero gsarti julien-c davidefiocco eliolio davanstrien wgpubs pierrci nouamanetazi abhishek raulmarindev iarbel AdjectiveAllison dariolopez datablets Chunte svilupp doolayer jackvial mishig cbensimon jeffboudier victor merve d0rj samusenps Yudiss sbrandeis

gabrielmbmb plaguss victor osanseviero radames dvilasuero julien-c davanstrien DamarJati abhishek raulmarindev Chunte mishig the-qa-company sbrandeis

dvilasuero julien-c davanstrien osanseviero raulmarindev Chunte mishig sbrandeis

kramp mwitiderrick raulmarindev Chunte MoritzLaurer mishig sbrandeis


lbourdois osanseviero pierrci santiviquez atulyaatul glenn-parham neovalle PetraLB gsarti mrfakename himanshubeniwal tunguz Felladrin mariagrandury jefferylovely Wauplin sandrinjoy thibaultM clefourrier radames sophiamyang alielfilali01 davanstrien abhishek ybelkada josers18 joeddav thomwolf DGoradia ceyda qqliangqi chansung rishiraj mwz shivance victor clem susnato lazarustda Javiai mcpotato Chunte themanas021 adarshxs Kirti jarvisx17 nouamanetazi lysandre TuringsSolutions Midgardsormr Tanvir1337 mishig stubbi sbrandeis

PetraLB osanseviero himanshubeniwal tunguz mariagrandury Wauplin radames sophiamyang DGoradia pain KnutJaegersberg beratcmn clem susnato Chunte themanas021 patrickfleith Midgardsormr Tanvir1337 stubbi

osanseviero sophiamyang pain porar18837 clem Chunte themanas021 himanshubeniwal

himanshubeniwal tunguz sophiamyang ybelkada clem Chunte themanas021


just setting up my new hf social posts account feature 🤗

jbilcke-hf linoyts osanseviero davanstrien pierrci Wauplin clefourrier radames gz888 KnutJaegersberg Chunte Outer-Spatial nouamanetazi clem VictorSanh abidlabs

mergisi Chunte Tonic clem rwightman abidlabs Epiculous