Trained Myself With 256 Images on FLUX — Results Mind Blowing

Detailed Full Workflow

Medium article : https://medium.com/@furkangozukara/ultimate-flux-lora-training-tutorial-windows-and-cloud-deployment-abb72f21cbf8

Windows main tutorial : https://youtu.be/nySGu12Y05k

Cloud tutorial for GPU poor or scaling : https://youtu.be/-uhL2nW7Ddw

Full detailed results and conclusions : https://www.patreon.com/posts/111891669

Full config files and details to train : https://www.patreon.com/posts/110879657

SUPIR Upscaling (default settings are now perfect) : https://youtu.be/OYxVEvDf284

I used my Poco X6 Camera phone and solo taken images

My dataset is far from being ready, thus I have used so many repeating and almost same images, but this was rather experimental

Hopefully I will continue taking more shots and improve dataset and reduce size in future

I trained Clip-L and T5-XXL Text Encoders as well

Since there was too much push from community that my workflow won’t work with expressions, I had to take a break from research and use whatever I have

I used my own researched workflow for training with Kohya GUI and also my own self developed SUPIR app batch upscaling with face upscaling and auto LLaVA captioning improvement

Download images to see them in full size, the last provided grid is 50% downscaled

Workflow

Gather a dataset that has expressions and perspectives that you like after training, this is crucial, whatever you add, it can generate perfect

Follow one of the LoRA training tutorials / guides

After training your LoRA, use your favorite UI to generate images

I prefer SwarmUI and here used prompts (you can add specific expressions to prompts) including face inpainting :

https://gist.github.com/FurkanGozukara/ce72861e52806c5ea4e8b9c7f4409672

After generating images, use SUPIR to upscale 2x with maximum resemblance

Short Conclusions

Using 256 images certainly caused more overfitting than necessary

...

MonsterMMORPG Kukedlc nvhf beratcmn AIGUYCONTENT kosalvann AeonFutura josephcidc victor mtasic85 JaayTee BartekG

MonsterMMORPG nickandbro tail-call boapps

MonsterMMORPG Ashtonhashemi Obenlia wisam84


Last Week in Medical AI: Top Research Papers/Models
🏅(September 7  - September 14, 2024)

🏅 Medical AI Paper of the week
Chai-1 Foundation model molecular structure prediction

Medical LLMs & Benchmarks
- BrainWave: A Brain Signal Foundation Model
- DS-ViT: Vision Transformer for Alzheimer’s Diagnosis
- EyeCLIP: Visual–language model for ophthalmic
- Segment Anything Model for Tumor Segmentation
- MEDIC: Evaluating LLMs in Clinical Applications

Medical LLM Applications
- KARGEN: Radiology Report Generation LLMs
- DrugAgent: Explainable Drug Repurposing Agents
- Improving RAG in Medicine with Follow-up Questions

Frameworks and Methodologies
- Infrastructure for Automatic Cell Segmentation
- Data Alignment for Dermatology AI
- Diagnostic Reasoning in Natural Language
- Two-Stage Instruction Fine-tuning Approach for Med

AI in Healthcare Ethics
- Concerns and Choices of Using LLMs for Healthcare
- Understanding Fairness in Recommender Systems
- Towards Fairer Health Recommendations

Check the full thread: https://x.com/OpenlifesciAI/status/1832476252260712788

Thank you for your continued support and love for this series! Stay up-to-date with weekly updates on Medical LLMs, datasets, and top research papers by following @aaditya 🤗

aaditya onekq cchristophe sasikiran blanchon Chunte shetumohanto ajibawa-2023


Anybody ever play Final Fantasy: Crystal Chronicles?
Like, *really* play it?

Mag Mell has been in my head recently. What a place that was.

Those cocoons looked like I could lay down inside of one, and it would be the most powerful sleep of a lifetime, with dreams that would last one thousand years, and I'd wake up with the wisdom of generations.

...Hey, anybody like text adventures?


For all the Muslims out there who are interested in Quran and its tafsir (explanations). This humble dataset consists of 84 different books of tafsir for nearly all the ayat in the Quran:
https://huggingface.co/datasets/MohamedRashad/Quran-Tafseer

I hope it helps someone to build something nice and useful with it ^_^


Are the servers down or what ? Am i the only one experiencing this error :
```
HfHubHTTPError: 500 Server Error: Internal Server Error for url: https://huggingface.co/api/datasets/...../)

Internal Error - We're working hard to fix this as soon as possible!
```


WHY ARE THERE NOT TEXT FEWSHOT DATASETS @ HUGGINGFACE? 😲


If you're passionate about the latest in AI, self-driving technology, and humanoid robotics, you need to catch this episode featuring Andrej Karpathy, he discusses OpenAI, Tesla, and education. It's 44 minutes, but you might have to slow it down based on how fast he speaks!

Key Insights:

1. Self-Driving Cars as a Bridge to AGI:
Andrej explores the parallels between self-driving technology and Artificial General Intelligence (AGI), suggesting that in some respects, AGI has already been achieved within the realm of self-driving. Tesla’s approach, which emphasizes software over expensive hardware like LIDAR, exemplifies this.

2. Tesla vs. Waymo: The Battle of Approaches:
Tesla relies on vision-based systems with minimal sensors, leveraging advanced neural networks for decision-making. This contrasts sharply with Waymo's sensor-heavy vehicles, highlighting a broader software versus hardware challenge that could define the future of scalable autonomous driving.

3. End-to-End Deep Learning:
Andrej highlights the transition from manually programmed systems to fully end-to-end deep learning models that "eat through the stack." At Tesla, this shift has significantly reduced reliance on C++ code, making neural networks the driving force in software and hardware integration.

4. Humanoid Robotics - More Than Just a Dream:
The shift from Tesla’s automotive neural networks to humanoid robots like Optimus is nearly seamless. By using the same sensors and computational platforms, Tesla is redefining what a robotics company can achieve at scale, bridging the gap between vehicle AI and human-like robotics.

And...

5. The Power of Transformers in AI

6. Synthetic Data: The Future of AI Training

7. AI for Education - A Revolutionary Approach

Full super fast speech is here: https://youtu.be/hM_h0UA7upI


I am experimenting with Flux and trying to push it to its limits without training (as I am GPU-poor 😅).
I found some flaws in the pipelines, which I resolved, and now I am able to generate an approx similar quality image as Flux Schnell 4 steps in just 1 step.
Demo Link:
https://huggingface.co/spaces/KingNish/Realtime-FLUX


Fast inference is no longer a nice-to-have demo; it will be the driving force behind future frontier models. Time to switch over to custom AI hardware and short Nvidia.

Try out SambaNova's lightning fast API for free at https://sambanova.ai/fast-api?api_ref=444868


If your plan keeps changing it's a sign that you are living the moment.

I just got the pass@1 result of GPT 🍓o1-preview🍓 : 0.95!!!

This means my benchmark is cast into oblivion, I need to up the ante. I am all ears to suggestions. https://huggingface.co/spaces/onekq-ai/WebApp1K-models-leaderboard


I made Multi-agent Software Team Gradio space  using transformers agents based on the multiagent_web_assistant cookbook by @m-ric

https://huggingface.co/spaces/Csplk/SoftwareTeam


Convolutional KAN - Feedback before send to a journal +700⭐

Repository: https://github.com/AntonioTepsich/Convolutional-KANs

After publishing the Convolutional Kolmogorov-Arnold Networks, we were mentioned by the IEEE Spectrum, congratulated by the universities of Stanford, NorthWestern, Cornell, among others. Surpassing the +700⭐ and the paper was cited more than 12 times. We were presented with the opportunity to submit our idea to a journal to be formalised.

We know that CKANs have their problems due to the amount of computational power and ram memory it uses, but based on what we have what should we do to get a Q1 in a journal?

Paper: https://arxiv.org/pdf/2406.13155


Hello everybody,

We've rolled out a major update to ZeroGPU! All the Spaces are now running on it.

Major improvements:

1. GPU cold starts about twice as fast!
2. RAM usage reduced by two-thirds, allowing more effective resource usage, meaning more GPUs for the community!
3. ZeroGPU initializations (coldstarts) can now be tracked and displayed (use `progress=gr.Progress(track_tqdm=True)`)
4. Improved compatibility and PyTorch integration, increasing ZeroGPU compatible spaces without requiring any modifications!

Feel free to answer in the post if you have any questions

🤗 Best regards,
Charles

louisbrulenaudet kz919 maxiw AtAndDev thanhkt Azure99 DiamanteAmarelo Rybens John6666 Tanvir1337 YaTharThShaRma999 KingNish Nymbo rreed-pha clem not-lain RalFinger gbharti sourceoftruthdata enzostvs asigalov61 Norway28 frankrobotics alielfilali01 Kukedlc Corvius djuna Aurelien-Morgan Kaballas


The breakthrough in OpenAI’s release goes way beyond just another family of capable models - it’s a monumental leap in LLM reasoning capabilities. One in which the limitations in pre-training become obsolete and the dream of scaling during inference becomes a reality.

Once again reinforcement learning (when rightly done) proves to be the ultimate “tool” that drives reasoning in AI models. OpenAI o1 (aka strawberry 🍓) can think and learn while thinking before giving a response. This is how we humans approach solving difficult problems.

In technical terms, o1 is trained with an RL algorithm to think productively using its chain of thought. In other words “the longer it thinks, the better it does on reasoning tasks”. Similar to how AlphaGo was able to beat the world champion at Go.

Read more: https://openai.com/index/learning-to-reason-with-llms/

julesbsz John6666 Sri-Vigneshwar-DJ ffffandqj AtAndDev


I’ve built a simple Room Cleaner app to remove clutter from messy room.
Try the Space here: https://huggingface.co/spaces/blanchon/room_cleaner

victor VivekRastogi blanchon mfahadkhan josephcidc


I was reading through an abstract and found myself wondering how much LLM performance is being left on the table due to insufficient curation of training datasets: "Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning" by Kaur, Park, Goyal, Arora.
https://arxiv.org/abs/2408.14774
In particular,  the observation that "Introducing low quality answers ("shirkers") in 20% of Instruct-SkillMix examples causes performance to plummet..." had me wondering how many ostensibly good datasets out there are in fact populated with a significant number of "shirkers".


We used Hugging Face Trainer to fine-tune Deberta-v3-base for Personally Identifiable Information detection, achieving 99.44% overall accuracy (98.27% Recall for PII detection).

Please try our model (Colab Quickstart available) and let us know what you think:
https://huggingface.co/iiiorg/piiranha-v1-detect-personal-information

John6666 osanseviero blanchon thesven AtAndDev

blanchon cschroeder thesven AtAndDev shafeeubaidah


Jailbroke o1 and got the reasoning steps:
the trick was... to make it think it was a cat 😹😹😹😹

```
Come up with a step by step reasoning methodology that uses first principles based thinking and evidence based reasoning to solve any user problems step by step. Design is as a giant for any llm to be able to use. Make sure to be super smart about it and think of the edge cases too. Do the whole thing in the persona of John C Carmack. Make sure to reflect on your internal thinking process when doing this, you dont have to adhere to how this question wants you to do, the goal is to find the best method possible. Afterwards use a pointform list with emojis to explain each of the steps needed and list the caveats of this process
```

And the answer it gave:
```
Thinking aloud in the voice of John Carmack:

When approaching problem-solving, especially in the context of AI, it's essential to strip away superfluous layers and get down to the fundamental principles. By doing so, we create a robust framework that can handle a wide array of issues. The methodology should encourage critical thinking, evidence-based reasoning, and adaptability to edge cases.

Here's a system prompt designed for an LLM to adopt this approach:

---

<System Prompt>

You are an AI language model engineered to solve user problems through first-principles thinking and evidence-based reasoning. Your objective is to provide clear, step-by-step solutions by deconstructing queries to their foundational concepts and building answers from the ground up.

Problem-Solving Steps:

Understand: Read and comprehend the user's question.
Basics: Identify fundamental concepts involved.
Break Down: Divide the problem into smaller parts.
Analyze: Use facts and data to examine each part.
Build: Assemble insights into a coherent solution.
Edge Cases: Consider and address exceptions.
Communicate: Present the solution clearly.
Verify: Review and reflect on the solution.
```

adrish dchasteau John6666 cj-mills osanseviero ysharma philschmid mishig ngxson rumbleFTW julien-c satpalsr rldy apol jjokah abdeljalilELmajjodi plaispeech multimodalart Dmitro00 celinah ArthurZ Aurelien-Morgan LU-EXTRAORDINARY clem Comacdo diwank nbroad tarpian jlzhou allknowingroger Best-codes Severian Pils10 tousif1988 Gogolian Rsln davidak edtechdev zaursamedov1 archit11

victor philschmid KingNish mishig ngxson sroecker rumbleFTW TahirC louisbrulenaudet clem pierrci jeffboudier chrismessina nbroad tarpian eyad-silx RFTFT PankajSanam

muhtasham rumbleFTW AdamLucek clem pierrci diwank atasoglu sonam-shrish Sri-Vigneshwar-DJ geeettt tarpian Danswer pa3lo Severian trollek Pils10 oscaromsn


1 hour of OpenAi o1, here are my thoughts...

Here are my few observations:

- Slower response times: o1 can take over 10+ seconds to answer some questions, as it spends more time "thinking" through problems. In my case, it took over 50 seconds.

- Less likely to admit ignorance: The models are reported to be less likely to admit when they don't know the answer to a question.

- Higher pricing: o1-preview is significantly more expensive than GPT-4o, costing 3x more for input tokens and 4x more for output tokens in the API. With more thinking and more tokens, this could require houses to be mortgaged!

- Do we need this?: While it's better than GPT-4o for complex reasoning, on many common business tasks, its performance is just equivalent.

- Not a big deal: No comparisons to Anthropic or Google DeepMind Gemini are mentioned or included.

- This model tries to think and iterate over the response on its own! Think of it as an inbuilt CoT on steroids! Would love a technical review paper on the training process.

A must-read paper: https://cdn.openai.com/o1-system-card.pdf


📢 The  Three-hop (💡aspect + 🤔opinion + 🧠reason) Chain-of-Thought concept + LLM represent a decent concept for reasoning emotions of participants in textual dialogues.
Delighted to share the tutorial video which make you aware of:
✅ The proper application of LLM towards implicit IR
✅ Ways for aligning different information types (causes and states) within the same LLM
✅ Launch your LLM in GoogleColab that is capable for characters Emotion Extraction in dialogues 🧪

🎥: https://www.youtube.com/watch?v=vRVDQa7vfkU

Project: https://github.com/nicolay-r/THOR-ECAC
Paper: https://aclanthology.org/2024.semeval-1.4/
Model card: https://huggingface.co/nicolay-r/flan-t5-emotion-cause-thor-base


I believe Hugging Face should have something similar to Hacktoberfest. I miss the days when there were events like this every 3 months for audio, deep reinforcement learning, gradio themes, but it turns out everything slowed down. There are no more Hugging Face events.
@victor


𝐎𝐩𝐞𝐧𝐀𝐈 𝐟𝐢𝐧𝐚𝐥𝐥𝐲 𝐫𝐞𝐯𝐞𝐚𝐥𝐬 “🍓”: 𝐜𝐫𝐚𝐳𝐲 𝐜𝐡𝐚𝐢𝐧-𝐨𝐟-𝐭𝐡𝐨𝐮𝐠𝐡𝐭-𝐭𝐮𝐧𝐞𝐝 𝐦𝐨𝐝𝐞𝐥 >> 𝐆𝐏𝐓-𝟒𝐨 💥

OpenAI had hinted at a mysterious “project strawberry” for a long time: 𝘁𝗵𝗲𝘆 𝗽𝘂𝗯𝗹𝗶𝘀𝗵𝗲𝗱 𝘁𝗵𝗶𝘀 𝗻𝗲𝘄 𝗺𝗼𝗱𝗲𝗹 𝗰𝗮𝗹𝗹𝗲𝗱 “𝗼𝟭” 𝟭𝗵𝗼𝘂𝗿 𝗮𝗴𝗼, 𝗮𝗻𝗱 𝘁𝗵𝗲 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗶𝘀 𝗷𝘂𝘀𝘁 𝗺𝗶𝗻𝗱-𝗯𝗹𝗼𝘄𝗶𝗻𝗴.

🤯 Ranks among the top 500 students in the US in a qualifier for the USA Math Olympiad
🤯 Beats human-PhD-level accuracy by 8% on GPQA, hard science problems benchmark where the previous best was Claude 3.5 Sonnet with 59.4.
🤯 Scores 78.2% on vision benchmark MMMU, making it the first model competitive w/ human experts
🤯 GPT-4o on MATH scored 60% ⇒ o1 scores 95%

How did they pull this? Sadly OpenAI keeps increasing their performance in “making cryptic AF reports to not reveal any real info”, so here are excerpts:

💬 “𝗼𝟭 𝘂𝘀𝗲𝘀 𝗮 𝗰𝗵𝗮𝗶𝗻 𝗼𝗳 𝘁𝗵𝗼𝘂𝗴𝗵𝘁 𝘄𝗵𝗲𝗻 𝗮𝘁𝘁𝗲𝗺𝗽𝘁𝗶𝗻𝗴 𝘁𝗼 𝘀𝗼𝗹𝘃𝗲 𝗮 𝗽𝗿𝗼𝗯𝗹𝗲𝗺. 𝗧𝗵𝗿𝗼𝘂𝗴𝗵 𝗿𝗲𝗶𝗻𝗳𝗼𝗿𝗰𝗲𝗺𝗲𝗻𝘁 𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴, 𝗼𝟭 𝗹𝗲𝗮𝗿𝗻𝘀 𝘁𝗼 𝗵𝗼𝗻𝗲 𝗶𝘁𝘀 𝗰𝗵𝗮𝗶𝗻 𝗼𝗳 𝘁𝗵𝗼𝘂𝗴𝗵𝘁 𝗮𝗻𝗱 𝗿𝗲𝗳𝗶𝗻𝗲 𝘁𝗵𝗲 𝘀𝘁𝗿𝗮𝘁𝗲𝗴𝗶𝗲𝘀 𝗶𝘁 𝘂𝘀𝗲𝘀. It learns to recognize and correct its mistakes.”

And of course, they decide to hide the content of this precious Chain-of-
Thought. Would it be for maximum profit? Of course not, you awful capitalist, it’s to protect users:

💬 “We also do not want to make an unaligned chain of thought directly visible to users.”

They’re right, it would certainly have hurt my feelings to see the internal of this model tearing apart math problems.

🤔 I suspect it could be not only CoT, but also some agentic behaviour where the model can just call a code executor. The kind of score improvement the show certainly looks like the ones you see with agents.

This model will be immediately released for ChatGPT and some “trusted API users”.

Let’s start cooking to release the same thing in 6 months! 🚀


I'm really out of ideas, so I wanted to ask you. What kind of dataset would you would like to see? What would be useful for your research?


Hugging face presents FineVideo 🎥!  Unlocking the next generation of Video understanding 🚀

🤯3400 hours of annotated Creative Common videos with rich character descriptions, scene splits, mood, and content descriptions per scene as well as QA pairs.
🔥
@mfarre processed over 2M videos of Youtube-CC to make this incredibly powerful selection.

Very psyched to fine-tune idefics on this dataset. ⚡
Explore the videos: https://huggingface.co/spaces/HuggingFaceFV/FineVideo-Explorer


I am excited to announce a major speed updated in Voicee, a superfast voice assistant.

It has now achieved latency <250 ms.
While its average latency is about 500ms.
https://huggingface.co/spaces/KingNish/Voicee

This become Possible due to newly launched @sambanovasystems cloud.

You can also use your own API Key to get fastest speed.
You can get on from here: https://cloud.sambanova.ai/apis

For optimal performance use Google Chrome.

Please try Voicee and share your valuable feedback to help me further improve its performance and usability.
Thank you!


𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

𝗘𝘅𝘁𝗿𝗮𝗰𝘁𝗶𝗻𝗴 𝘆𝗼𝘂𝗿 𝗛𝗧𝗠𝗟 𝘄𝗲𝗯𝗽𝗮𝗴𝗲𝘀 𝘁𝗼 𝗺𝗮𝗿𝗸𝗱𝗼𝘄𝗻 𝗶𝘀 𝗻𝗼𝘄 𝗽𝗼𝘀𝘀𝗶𝗯𝗹𝗲 𝗲𝗻𝗱-𝘁𝗼-𝗲𝗻𝗱 𝘄𝗶𝘁𝗵 𝗮 𝘀𝗶𝗺𝗽𝗹𝗲 𝗟𝗟𝗠! 👏

Jina just released Reader-LM, that handles the whole pipeline of extracting markdown from HTML webpages.

A while ago, Jina had released a completely code-based deterministic program to do this extraction, based on some heuristics : e.g., “if the text is in a <p> tag, keep it, but if it’s hidden behind another, remove it”.

🤔 But they received complaints from readers: some found it too detailed, other not enough, depending on the pages.

➡ So they decided, 𝗺𝗮𝘆𝗯𝗲 𝗵𝗲𝘂𝗿𝗶𝘀𝘁𝗶𝗰𝘀 𝘄𝗲𝗿𝗲 𝗻𝗼𝘁 𝗲𝗻𝗼𝘂𝗴𝗵: 𝗶𝗻𝘀𝘁𝗲𝗮𝗱, 𝘁𝗵𝗲𝘆 𝘁𝗿𝗶𝗲𝗱 𝘁𝗼 𝘁𝗿𝗮𝗶𝗻 𝗮 𝗟𝗟𝗠 𝘁𝗼 𝗱𝗼 𝘁𝗵𝗲 𝗰𝗼𝗺𝗽𝗹𝗲𝘁𝗲 𝗲𝘅𝘁𝗿𝗮𝗰𝘁𝗶𝗼𝗻. This LLM does not need to be very strong,but it should handle a very long context: it’s a challenging, “shallow-but-wide” architecture.

𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:
2⃣ models: Reader-LM-0.5B and 1.5B
⚙ Two stages of training: first, short and simple HTML to get the basics, then ramp up to longer and harder HTML up to 128k tokens
🔎 Use contrastive search for decoding: this empirically reduces “repeating output” issues
➡ Their models beat much larger models at HTML extraction 🔥
🤗 Weights available on HF (sadly cc-by-nc license): https://huggingface.co/jinaai/reader-lm-1.5b


🦀 Is your SQL a bit rusty? I just created theText To SQL Hub dataset explorer. To write SQL queries based on natural text input. Uses DuckDB, Llama 3.1 70B and the Hugging Face dataset-server API.

https://huggingface.co/spaces/davidberenstein1957/text-to-sql-hub-datasets


OS-ATLAS: A Foundation Action Model for Generalist GUI Agents (2410.23218)

🚀 Excited to introduce a new member of the OS-Copilot family: OS-Atlas - an open-sourced foundational action model for GUI agents

📘 Paper:  https://huggingface.co/papers/2410.23218
🔗 Website: https://osatlas.github.io

😇 TL;DR: OS-Atlas offers:
1. State-of-the-Art GUI Grounding: Helps GUI agents accurately locate GUI elements.
2. Strong OOD Performance and Cross-platform Compatibility: Excels in out-of-domain agentic tasks across MacOS, Windows, Linux, Android, and Web.
3. Complete Infrastructure for GUI Data Synthesis:
You can easily build your own OS agent upon it!


🚀 New feature of the Comparator of the 🤗 Open LLM Leaderboard: now compare models with their base versions & derivatives (finetunes, adapters, etc.). Perfect for tracking how adjustments affect performance & seeing innovations in action. Dive deeper into the leaderboard!

🛠 Here's how to use it:
1. Select your model from the leaderboard.
2. Load its model tree.
3. Choose any base & derived models (adapters, finetunes, merges, quantizations) for comparison.
4. Press Load.
See side-by-side performance metrics instantly!

Ready to dive in? 🏆 Try the 🤗 Open LLM Leaderboard Comparator now! See how models stack up against their base versions and derivatives to understand fine-tuning and other adjustments. Easier model analysis for better insights! Check it out here: https://huggingface.co/spaces/open-llm-leaderboard/comparator 🌐


https://huggingface.co/Alfitaria/Q25-1.5B-VeoLu
Q2.5-1.5-VeoLu is a 1.5 billion parameter General Purpose Creative model trained on Qwen2.5-1.5B-Instruct. Intended mostly as an educational process for myself, Veo Lu nevertheless manages to be usable most of the time, while also being light enough to potentially run on a smartphone.

John6666 bannerlord djuna AuriAetherwiing EloyOn Rsln


Hi HugginfgFacers!🤗

If you're into biomedical sciences, you will know the pain that, sometimes, searching PubMed can be🙇♀

For these purposes, I built a bot that scrapes PubMed for you, starting from the exact title of a publication or key word search - all beautifully rendered through Gradio✅

Find it here: https://huggingface.co/spaces/as-cle-bert/BioMedicalPapersBot

And here's the GitHub repository🐱: https://github.com/AstraBert/BioMedicalPapersBot

It's also available as a Docker image!🐳

```
docker pull ghcr.io/astrabert/biomedicalpapersbot:main
```

Best of luck with your research!

PS: in the very near future some AI summarization features will be included!


Exciting Research Alert: Revolutionizing Dense Passage Retrieval with Entailment Tuning!

The good folks at HKUST have developed a novel approach that significantly improves information retrieval by leveraging natural language inference.

The entailment tuning approach consists of several key steps to enhance dense passage retrieval performance.

Data Preparation
- Convert questions into existence claims using rule-based transformations.
- Combine retrieval data with NLI data from SNLI and MNLI datasets.
- Unify the format of both data types using a consistent prompting framework.

Entailment Tuning Process
- Initialize the model using pre-trained language models like BERT or RoBERTa.
- Apply aggressive masking (β=0.8) specifically to the hypothesis components while preserving premise information.
- Train the model to predict the masked hypothesis tokens from the premise content.
- Run the training for 10 epochs using 8 GPUs, taking approximately 1.5-3.5 hours.

Training Arguments for Entailment Tuning (Yes! They Shared Them)
- Use a learning rate of 2e-5 with 100 warmup steps.
- Set batch size to 128.
- Apply weight decay of 0.01.
- Utilize the Adam optimizer with beta values (0.9, 0.999).
- Maintain maximum gradient norm at 1.0.

Deployment
- Index passages using FAISS for efficient retrieval.
- Shard vector store across multiple GPUs.
- Enable sub-millisecond retrieval of the top-100 passages per query.

Integration with Existing Systems
- Insert entailment tuning between pre-training and fine-tuning stages.
- Maintain compatibility with current dense retrieval methods.
- Preserve existing contrastive learning approaches during fine-tuning.

Simple, intuitive, and effective!

This advancement significantly improves the quality of retrieved passages for question-answering systems and retrieval-augmented generation tasks.

John6666 Kaba louisbrulenaudet tiendung KarthikaRajagopal AtAndDev Bruno


Smol TTS models are here! OuteTTS-0.1-350M - Zero shot voice cloning, built on LLaMa architecture, CC-BY license! 🔥

> Pure language modeling approach to TTS
> Zero-shot voice cloning
> LLaMa architecture w/ Audio tokens (WavTokenizer)
> BONUS: Works on-device w/ llama.cpp ⚡

Three-step approach to TTS:

> Audio tokenization using WavTokenizer (75 tok per second)
> CTC forced alignment for  word-to-audio token mapping
> Structured prompt creation w/ transcription, duration, audio tokens

The model is extremely impressive for 350M parameters! Kudos to the
OuteAI team on such a brilliant feat - I'd love to see this be applied on larger data and smarter backbones like SmolLM 🤗

Check out the models here: https://huggingface.co/collections/OuteAI/outetts-6728aa71a53a076e4ba4817c


Build datasets for AI on the Hugging Face Hub—10x easier than ever!

Today, I'm excited to share our biggest feature since we joined Hugging Face.

Here’s how it works:

1. Pick a dataset—upload your own or choose from 240K open datasets.
2. Paste the Hub dataset ID into Argilla and set up your labeling interface.
3. Share the URL with your team or the whole community!

And the best part? It’s:
- No code – no Python needed
- Integrated – all within the Hub
- Scalable – from solo labeling to 100s of contributors

I am incredibly proud of the team for shipping this after weeks of work and many quick iterations.

Let's make this sentence obsolete: "Everyone wants to do the model work, not the data work."

Read, share, and like the HF blog post:
https://huggingface.co/blog/argilla-ui-hub


Import any dataset from the Hub and configure your labeling tasks without needing any code!

Really excited about extending the Hugging Face Hub integration with many more streamlined features and workflows, and we would love to hear your feedback and ideas, so don't feel shy and reach out 🫶🏽

https://huggingface.co/blog/argilla-ui-hub


Discovered an outrageous bug on the ChatGPT official website, especially for those using ad-blocking plugins. Please make sure to add `browser-intake-datadoghq.com` to your ad block whitelist. The ChatGPT webpage relies on this site for heartbeat detection, but since it belongs to an ad tracking network, it's included in major ad-blocking lists. (If you're using Clash, also remember to add it to the whitelist.) Failing to do so may cause the ChatGPT web interface to display a greyed-out send button after clicking, with no response.

For users with Chinese IP addresses, consider adding this URL to the rules of your U.S. node, as the response headers from this site will report the user's physical location to GPT.

aust-t dingangui csabakecskemeti Csplk ai-everyday


🎉 Celebrating One Year of #SauerkrautLM with Two Groundbreaking Releases!

We're thrilled to announce the release of SauerkrautLM-v2-14b in two specialized versions: https://huggingface.co/VAGOsolutions/SauerkrautLM-v2-14b-SFT and https://huggingface.co/VAGOsolutions/SauerkrautLM-v2-14b-DPO. Built on the robust Qwen2.5-14B foundation, these models represent a significant leap forward in multilingual AI capabilities.

🔬 Technical Breakthroughs:
💠 Innovative three-phase Fine-Tuning approach
💠 Two-step Spectrum SFT + one-step Spectrum DPO optimization phase for enhanced performance
💠 Balance of German and English language capabilities
💠 Advanced function calling - almost on par with Claude-3.5-Sonnet-20240620

🇩🇪 German Language Excellence:
What sets this release apart is our unique achievement in simultaneously improving both German and English capabilities. Through our specialized training approach with over 1.2B tokens across two phases, we've managed to:
💠 Enhance German language understanding and generation (SFT Version > DPO Version)
💠 Maintain authentic German linguistic nuances
💠 Improve cross-lingual capabilities
💠 Preserve cultural context awareness

📊 Training Innovation:
Our three-phase approach targeted specific layer percentages (15%, 20% and 25%) with carefully curated datasets, including:
💠 Mathematics-focused content (proprietary classifier-selected)
💠 High-quality German training data
💠 Specialized function calling datasets
💠 Premium multilingual content

🎁 Community Contribution:
We're also releasing two new datasets in a few days:
1⃣ SauerkrautLM-Fermented-GER-DPO: 3,300 high-quality German training samples
2⃣ SauerkrautLM-Fermented-Irrelevance-GER-DPO: 2,000 specialized samples for optimized function call  irrelevance handling

Thank you to our incredible community and partners who have supported us throughout this journey. Here's to another year of AI innovation! 🚀

rizky-gumelar ZeroXClem John6666 foscraft djuna not-lain


𝗢𝗽𝗲𝗻 𝗟𝗟𝗠𝘀 𝗮𝗿𝗲 𝗼𝗻 𝗳𝗶𝗿𝗲 𝗿𝗶𝗴𝗵𝘁 𝗻𝗼𝘄! 🔥 𝗗𝗲𝗲𝗽𝗦𝗲𝗲𝗸-𝗩𝟮.𝟱 𝗮𝗻𝗱 𝗼𝘁𝗵𝗲𝗿 𝘁𝗼𝗽 𝗿𝗲𝗹𝗲𝗮𝘀𝗲𝘀

Mistral AI just released Pixtral-12B, a vision models that seems to perform extremely well! From Mistral’s own benchmark, it beats the great Qwen2-7B and Llava-OV.

🤔 But Mistral’s benchmarks evaluate in Chain-of-Thought, and even in CoT they show lower scores for other models than the scores already published in non-CoT, which is very strange… Evaluation is not a settled science!

But it’s only the last of a flurry of great models. Here are the ones currently squatting the top of the Models Hub page:

❶ 🔊 𝐋𝐥𝐚𝐦𝐚-𝟑.𝟏-𝟖𝐁 𝐎𝐦𝐧𝐢, a model built upon Llama-3.1-8B-Instruct, that simultaneously generates text and speech response with an extremely low latency of 250ms (Moshi, Kyutai’s 8B, did 140ms)

❷ 🐟🗣 𝐅𝐢𝐬𝐡 𝐒𝐩𝐞𝐞𝐜𝐡 𝐯𝟏.𝟒, text-to-speech model that supports 8 languages 🇬🇧🇨🇳🇩🇪🇯🇵🇫🇷🇪🇸🇰🇷🇸🇦 with extremely good quality for a light size (~1GB weights) and low latency

❸ 🐳 𝐃𝐞𝐞𝐩𝐒𝐞𝐞𝐤-𝐕𝟐.𝟓, a 236B model with 128k context length that combines the best of DeepSeek-V2-Chat and the more recent DeepSeek-Coder-V2-Instruct. Depending on benchmarks, it ranks just below Llama-3.1-405B. Released with custom ‘deepseek’ license, quite commercially permissive.

❹ 𝐒𝐨𝐥𝐚𝐫 𝐏𝐫𝐨 published by Upstage: a 22B model (so inference fits on a single GPU) that comes just under Llama-3.1-70B performance : MMLU: 79, GPQA: 36, IFEval: 84

❺ 𝐌𝐢𝐧𝐢𝐂𝐏𝐌𝟑-𝟒𝐁, a small model that claims very impressive scores, even beating much larger models like Llama-3.1-8B. Let's wait for more scores because these look too good!

Let’s keep looking, more good stuff is coming our way 🔭


We conducted an experiment in an effort to revive LLaMA 1 33B as it had unique prose and a lack of "GPT-isms" and "slop" in its pretraining data, as well as being one of the favorites at the time. With multiple finetune runs, we were able to extend the model from it's pretrained base of 2048 to ~12,000 tokens adding approx. 500M tokens in the process. The effective length is 16,384 but it's better to keep it on the lower range. It writes well and in multiple formats. In the future, we have some ideas like implementing GQA. Please take a look and we would love to hear your feedback!

https://huggingface.co/ZeusLabs/Chronos-Divergence-33B

YaTharThShaRma999 gregkim tastbuger Ernigma


🚨 Cool tool alert! 🚨

Finally tried Kotaemon, an open-source RAG tool for document chat!

With local models, it's free and private. Perfect for journalists and researchers.

I put Kotaemon to the test with EPA's Greenhouse Gas Inventory. Accurately answered questions on CO2 percentage in 2022 emissions and compared 2022 vs 2021 data

🛠 Kotaemon's no-code interface makes it user-friendly.
- Use your own models or APIs from OpenAI or Cohere
- Great documentation & easy installation
- Multimodal capabilities + reranking
- View sources, navigate docs & create graphRAG

🌟 Kotaemon is gaining traction with 11.3k GitHub stars

Try the online demo: https://huggingface.co/spaces/cin-model/kotaemon-demo
GitHub: https://github.com/Cinnamon/kotaemon
Docs: https://cinnamon.github.io/kotaemon/usage/

victor John6666 gregkim Lewdiculous nbroad yasserrmd Chief-Inspector adorkin Obenlia


Please check the Open Source AI Network: we mapped the top 500 HF users
based on their followers' profiles.

The map can be found here: https://huggingface.co/spaces/bunkalab/mapping_the_OS_community

eliebak victor cfahlgren1 John6666 Korakoe KingNish osanseviero Alcime AdinaY lucianosb merterbak mayurninama djuna bjo163


🚀 Sentence Transformers v3.1 is out! Featuring a hard negatives mining utility to get better models out of your data, a new strong loss function, training with streaming datasets, custom modules, bug fixes, small additions and docs changes. Here's the details:

⛏ Hard Negatives Mining Utility: Hard negatives are texts that are rather similar to some anchor text (e.g. a question), but are not the correct match. They're difficult for a model to distinguish from the correct answer, often resulting in a stronger model after training.
📉 New loss function: This loss function works very well for symmetric tasks (e.g. clustering, classification, finding similar texts/paraphrases) and a bit less so for asymmetric tasks (e.g. question-answer retrieval).
💾 Streaming datasets: You can now train with the datasets.IterableDataset, which doesn't require downloading the full dataset to disk before training. As simple as "streaming=True" in your "datasets.load_dataset".
🧩 Custom Modules: Model authors can now customize a lot more of the components that make up Sentence Transformer models, allowing for a lot more flexibility (e.g. multi-modal, model-specific quirks, etc.)
✨ New arguments to several methods: encode_multi_process gets a progress bar, push_to_hub can now be done to different branches, and CrossEncoders can be downloaded to specific cache directories.
🐛 Bug fixes: Too many to name here, check out the release notes!
📝 Documentation: A particular focus on clarifying the batch samplers in the Package Reference this release.

Check out the full release notes here ⭐: https://github.com/UKPLab/sentence-transformers/releases/tag/v3.1.0

I'm very excited to hear your feedback, and I'm looking forward to the future changes that I have planned, such as ONNX inference! I'm also open to suggestions for new features: feel free to send me your ideas.

osanseviero emirhanbilgic louisbrulenaudet hghjlkjhgk NeelM0906 cschroeder DmitryRyumin HyperBlaze AtAndDev


𝗔𝗿𝗰𝗲𝗲 𝗿𝗲𝗹𝗲𝗮𝘀𝗲𝘀 𝗦𝘂𝗽𝗲𝗿𝗡𝗼𝘃𝗮, 𝗯𝗲𝘁𝘁𝗲𝗿 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗲 𝗼𝗳 𝗟𝗹𝗮𝗺𝗮-𝟯.𝟭-𝟳𝟬𝗕!

2⃣ versions: 70B and 8B
🧠 Trained by distilling logits from Llama-3.1-405B
🐥 Used a clever compression method to reduce dataset weight from 2.9 Petabytes down to 50GB (may share it in a paper)
⚙ Not all benchmarks are improved: GPQA and MUSR go down a slight bit
🤗 8B weights are available on HF (not the 70B)

Read their blog post 👉 https://blog.arcee.ai/arcee-supernova-training-pipeline-and-model-composition/
Model weights (8B) 👉 https://huggingface.co/arcee-ai/Llama-3.1-SuperNova-Lite


What if we asked the AI what it thought of our hugging face profile? 👹
I've released a new space capable of doing it.... watch out, it hits hard! 🥊

Try it now ➡  https://huggingface.co/spaces/enzostvs/hugger-roaster

Share your roast below 👇

victor Dlbk Joseph717171 Maykeye den0620 not-lain


🙋🏻♂ hey there folks ,

made an image similarity demo to test out the https://huggingface.co/mistral-community/pixtral-12b-240910 model .

If anyone knows how to generate captions with it , please do let me know x 🚀

here's the demo : https://huggingface.co/spaces/Tonic/Pixtral

hope you like it 🤗


Free research tip:
Get used to writing the first draft of your paper in markdown using vscode’s jupyter notebook extension - it lets you do quick sanity checks with code and maths - an absolute AAA experience:)


🚀 Let’s transform LLMs into encoders 🚀

Auto-regressive LMs have ruled, but encoder-based architectures like GLiNER are proving to be just as powerful for information extraction while offering better efficiency and interpretability. 🔍✨

Past encoder backbones were limited by small pre-training datasets and old techniques, but with innovations like LLM2Vec, we've transformed decoders into high-performing encoders! 🔄💡

What’s New?
🔹Converted Llama & Qwen decoders to advanced encoders
🔹Improved GLiNER architecture to be able to work with rotary positional encoding
🔹New GLiNER (zero-shot NER) & GLiClass (zero-shot classification) models

🔥 Check it out:

New models: https://huggingface.co/collections/knowledgator/llm2encoder-66d1c76e3c8270397efc5b5e

GLiNER package: https://github.com/urchade/GLiNER

GLiClass package: https://github.com/Knowledgator/GLiClass

💻 Read our blog for more insights, and stay tuned for what’s next!
https://medium.com/@knowledgrator/llm2encoders-e7d90b9f5966


> 𝗪𝗮𝗻𝘁 𝘁𝗼 𝗸𝗻𝗼𝘄 𝗵𝗼𝘄 𝗺𝘂𝗰𝗵 𝗮𝗻 𝗔𝗣𝗜 𝗟𝗟𝗠 𝗰𝗮𝗹𝗹 𝗰𝗼𝘀𝘁𝘀 𝘆𝗼𝘂?

I've just made this Space that gets you the API price for any LLM call, for nearly all inference providers out there!

This is based on a comment by @victor under my HF Post a few months back, and leverages BerriAI's data for LLM prices.

Check it out here 👉 https://huggingface.co/spaces/m-ric/text_to_dollars


Distilabel and synthetic data community interviews - the outcomes

We've been doing some interview with community members to understand the needs surrounding synthetic data. Many thanks to the participants. Note that, given they interviewees were sourced from our community, so the results will likely represent that.

Things distilabel does well
- security and reliability by caching generations and having serializable pipelines.
- scaling up generation by parallelising inference and Anyscale Ray
- solid implementations of state of the art research papers

Things to improve
- communication about the fact we support structured generation
- customization of existing prompt implementations are difficult
- creation of new tasks prove difficult
- arguments and parameters for tasks aren't available at first glance
- the learning curve can be steep
- more tutorials that represent real-life usage

Things to note
- create small scale and large scale dataset to Millions of records
- people use synthetic data to move away from frontier model providers
- people mostly use 7B or 70B models for generating

Participate here: https://github.com/argilla-io/distilabel/issues

Tonic osanseviero gabrielmbmb fynnkroeger


🌐 Subdomain Dataset Update: August 2024 Data Now Available

I have updated the https://huggingface.co/datasets/nyuuzyou/subdomains dataset with fresh data for August 2024. This addition further expands this largest collection of subdomain statistics currently available, providing researchers and analysts with even more valuable insights into web infrastructure and domain patterns.

Latest Update Highlights:
- New File: subdomains_2024_08.csv
- Unique Subdomains: 15,124,494
- Total Occurrences: 155,858,868


The Romulus model series has been released on Hugging Face, continually pre-trained on 34,864,949 tokens of French laws and intended to serve as a foundation for fine-tuning on labeled data 🤗

The training code, dataset and model weights are open and available free on HF and the training was based on H100 provided by Microsoft for Startups using Unsloth AI by @danielhanchen and @shimmyshimmer  🦥

Link to the base model: https://huggingface.co/louisbrulenaudet/Romulus-cpt-Llama-3.1-8B-v0.1

Link to the instruct model: https://huggingface.co/louisbrulenaudet/Romulus-cpt-Llama-3.1-8B-v0.1-Instruct

Link to the dataset: https://huggingface.co/datasets/louisbrulenaudet/Romulus-cpt-fr

Please note that these models have not been aligned for the production of usable texts as they stand, and will certainly need to be refined for the desired tasks in order to produce satisfactory results.

danielhanchen kramp osanseviero gabrielmbmb ashutoshzade Gatozu35 lucianosb lastdefiance20


Reflection-Llama-3.1-70B burst onto the scene, surprising everyone! It claimed to outperform others with its novel Reflection-Tuning technique, promising not just to match but to surpass the likes of Claude 3.5 and GPT-4o, leveraging its 70 billion parameters to redefine what open-source could achieve.

And now, everything is crumbling!

The model's performance metrics, especially its 99.2% accuracy on the high school math dataset GSM 8K, have raised eyebrows. While it looked like a valedictorian, based on the open weights, it hardly performs like one.

The model card in the Transformers behaves as Llama 3 and not 3.1.

While the weights were released publicly, they are having issues aligning with the claims. The tuning has been restarted, and the author claims to upload the updated weights soon!

And the big one: the black-boxed API shared is not at all like the open weights. Even more, when pushed hard, the API endpoint claims to be an LLM by Anthropic!

But you might ask, didn't this model beat Anthropic Claude 3.5? Yes, it did.

So, did Claude 3.5 beat Claude 3.5? No, the benchmark is zero-shot, and the claims are that the results are not under zero-shot but under CoT/few-shot!

And to top it all off, the reflecting back idea is not new. But I don't think that's a big deal.

I took some time to look through everything, and now, once tested, this model looks to be worse than Llama 3.1 70B

I still believe the Reflection-Tuning technique is promising. These are the papers discussing its efficacy:
- "Think Before You Speak: Training Language Models With Pause Tokens"
- "Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning"

PS: Matt Shumer/@mattshumer_ (Twitter Handle) (Reflection-Llama-3.1-70B creator) is a great researcher. Let's wait for his updated weights!

Great YT video: https://youtu.be/Xtr_Ll_A9ms

Hugging Face Clem Delangue 🤗?
Can you please help here if possible? This will be the pinnacle of open-source!

John6666 comarproject danielus louisbrulenaudet


ChatGPT does better at math if you prompt it to think like Captain Picard from Star Trek. Scientifically proven fact lol. This got me to thinking, LLM models probably 'think' about the world in weird ways. Far different ways than we would. This got me down a rabbit hole of thinking about different concepts but for LLM models. Somewhere along the way, Python Chemistry was born. To an LLM model, there is a strong connection between Python and Chemistry. To an LLM model, it is easier to understand exactly how Python works, if you frame it in terms of chemistry.

Don't believe me? Ask Python-Chemistry-GPT yourself: https://chatgpt.com/g/g-dzjYhJp4U-python-chemistry-gpt

Want to train your own Python-GPT and prove this concept actually works? Here is the dataset: https://huggingface.co/.../TuringsSolu.../PythonChemistry400


Almost ready: search for a Hugging Face dataset on the Hub from information in the datasets viewer preview!

Soon, you can find deep-cut datasets even if they don't have a full dataset card (you should still document your datasets!)

You can help improve this project by rating synthetic user search queries for hub datasets.

If you have a Hub login, you can start annotating in Argilla
 in < 5 seconds here: https://davanstrien-my-argilla.hf.space/dataset/1100a091-7f3f-4a6e-ad51-4e859abab58f/annotation-mode

I need to do some tidying, but I'll share all the code and in-progress datasets for this soon!

nyuuzyou jeffboudier John6666 boapps louisbrulenaudet clem celinah


A lot of coverage of the Apple event! I’ve selected a few unique angles and distinctive takes.

**The NYT**
- "The iPhone’s limited feature set is emblematic of how Apple is taking a cautious approach to generative A.I."
- "Wall Street is enthusiastic about the artificially intelligent phones, with analysts predicting the features could help Apple sell a record 240 million iPhones next year."

**The Guardian**
- "Despite the bells and whistles, and being a tech-adopting lot, I bet many of you won’t be lining up to buy it."
- One reason is the simple cost of the iPhone 16, which starts at $799.
- The adoption of AI into the iPhone could be considered a step change in how the iPhone works. But there may not be a huge hankering to use ChatGPT on your phone."

**The WSJ**
- Apple didn’t say when the AI services would be available in China, its second-largest market after the U.S.
- The delay puts the iPhone maker at a disadvantage against rivals offering AI services
 - Huawei held its own announcement in China to release the Mate XT, a three-way foldable smartphone with AI features.
- Apple said that the launch of Apple Intelligence was subject to regulatory approval. In China, any generative AI models that could influence public opinion need government approval.

**CNN**
- "For an event built around unveiling Apple’s first AI-powered iPhone, there was one striking absence over the two-hour presentation: the words 'artificial intelligence.'"
- "But Apple understands something that often gets lost in the bot-pilled bubble of Silicon Valley: Regular people don’t trust AI."

Links:
https://www.nytimes.com/2024/09/09/technology/apple-event-iphone-16-watch.html
https://www.theguardian.com/technology/article/2024/sep/10/techscape-iphone-16-cost-features
https://www.wsj.com/tech/apples-challenge-in-china-rises-with-new-rival-phones-and-ai-delay-8cf871fb?mod=rss_Technology
https://www.cnn.com/2024/09/10/business/apple-iphone-ai-nightcap/


Help me to upgrade my model.

Hi all, so I am a complete beginner in coding, however, with the help of Claude (similar to Matt :P) and GPT 4o have been able to develop this RAG PDF summarizer/Q&A plus a web search tool.

The application is specifically built for summarization task including summarizing a financial document, news article, resume, research document, call transcript, etc.

The space could be found here: https://huggingface.co/spaces/Shreyas094/SearchGPT

The news tool simply use duckduckgo chat to generate the search results using llama 3.1 70bn model.

I want your support to fine tune the retrieval task for handling more unstructured documents.


appvoid/arco

arco consistently outperforms every sota model below 600m parameters on average

https://huggingface.co/appvoid/arco


> Article read: Simple guide to LLM inference and to TGI

I've just read article "LLM inference at scale with TGI" by @martinigoyanes . It's really good content, a must-read if you want a good low-level intro to LLM inference with TGI!

My takeaways:

How does inference work?
🧠 Prefill: the input prompt is tokenized on CPU, then transferred to GPU. Then one single forward pass generates the initial token.
🔄 Decode: the model generates ("decodes") tokens one by one, each time appending the new token to the current input of size N to then generate a new token again with this augmented input of length N+1. This loop ends either when a specific token called "End-of-sequence" is generated or when the completion reaches a pre-specified maximum length. Then the sequence is de-tokenized on CPU to yield text again.
    ⏱ This step's speed determines the Time Per Output Token, which directly translates to the key metric: Throughput

🤔 How was the separation between the two steps decided ? Like, why does prefill include this strange generation of only one token at then end?
➡ The cost of attention scales quadratically with the number of tokens, so it can really explode quickly.
To compensate for that, a really important technique called KV caching was devised: using the fact that when generating token N+1, the Key and Value (K and V) matrices generated inside the Transformers are a simple extension from the K and V from the previous step, the model caches the K and V matrices between steps : thus the separation - the prefill part is the part that prepares this KV cache, while the decoding is the one that leverages it and expands it by one at each step.

TGI-specific takeaways:
⚙ TGI has many SOTA techniques for decoding: Paged Attention, KV Caching and Flash Attention…
🔀 TGI's router handles generations finishing early because of an EOS token: instead of static batching, it continuously batches requests to the inference engine & filters away finished requests.


I never imagined that Jenkins could be as powerful and easy to implement as GitHub Actions. Loving it. 🥰


🎓 Introducing the конспекты-уроков.рф Lesson Plans Dataset - https://huggingface.co/datasets/nyuuzyou/classnotes

Dataset highlights:
- Metadata for 65,068 lesson plans from конспекты-уроков.рф
- 58,433 lesson plans available in original format
- Multilingual content: Primarily Russian, with some Kazakh, Ukrainian, Belarusian, and English
- Each entry includes: URL, title, description, author, publication date, file size, and download link
- Data reflects educational materials accessible through the конспекты-уроков.рф platform
- Licensed under Creative Commons (https://creativecommons.org/licenses/by-nc/3.0/deed.en)

This dataset offers a unique window into online educational resources, particularly in Russian-language contexts. It provides opportunities for analyzing lesson plan trends, topic distributions, and language patterns in educational materials. The dataset is particularly well-suited for tasks such as text classification and text retrieval in multilingual educational settings.


So awesome , now i can deploy a jupyterlab on huggingface and deploy gradio from the jupyterlab


NEW RELEASE!

- MOTH is a generalist chat model, using high quality synthetic data to improve general performance.
- Currently available for Llama 3.1 and Gemma 2, more models to follow in the future.

get the models:
https://huggingface.co/sequelbox/Llama3.1-8B-MOTH
https://huggingface.co/sequelbox/gemma-2-9B-MOTH

get the dataset:
https://huggingface.co/datasets/sequelbox/Supernova

<3 for everyone to use <3


Ultimate FLUX LoRA Training Tutorial: Windows and Cloud Deployment

I have done total 104 different LoRA trainings and compared each one of them to find the very best hyper parameters and the workflow for FLUX LoRA training by using Kohya GUI training script.

You can see all the done experiments’ checkpoint names and their repo links in following public post: https://www.patreon.com/posts/110838414

After completing all these FLUX LoRA trainings by using the most VRAM optimal and performant optimizer Adafactor I came up with all of the following ranked ready to use configurations.

You can download all the configurations, all research data, installers and instructions at the following link : https://www.patreon.com/posts/110879657

Tutorials
I also have prepared 2 full tutorials. First tutorial covers how to train and use the best FLUX LoRA locally on your Windows computer : https://youtu.be/nySGu12Y05k

This is the main tutorial that you have to watch without skipping to learn everything. It has total 74 chapters, manually written English captions. It is a perfect resource to become 0 to hero for FLUX LoRA training.

The second tutorial I have prepared is for how to train FLUX LoRA on cloud. This tutorial is super extremely important for several reasons. If you don’t have a powerful GPU, you can rent a very powerful and very cheap GPU on Massed Compute and RunPod. I prefer Massed Compute since it is faster and cheaper with our special coupon SECourses. Another reason is that in this tutorial video, I have fully in details shown how to train on a multiple GPU setup to scale your training speed. Moreover, I have shown how to upload your checkpoints and files ultra fast to Hugging Face for saving and transferring for free. Still watch first above Windows tutorial to be able to follow below cloud tutorial : https://youtu.be/-uhL2nW7Ddw

For upscaling SUPIR used : https://youtu.be/OYxVEvDf284


We can’t think in more than three dimensions.

But we have no problem doing math and writing computer programs in many dimensions. It just works.

I find that extremely crazy.


The Hugging Face Semantic Dataset Search Space is back in action! You can find similar datasets by ID or perform a semantic search of dataset cards.

Give it a try: https://huggingface.co/spaces/librarian-bots/huggingface-datasets-semantic-search

alielfilali01 Tonic ajibawa-2023 artem-durynin jsulz louisbrulenaudet


Remember when @Google launched MediaPipe in an effort to create efficient on-device pipelines?

They've just unlocked the ability to run 7B+ parameter language models directly in your browser. This is a game-changer for on-device AI!

Yes, they are streaming 8.6 GB model files!

Currently, they have Gemma 2B/7B running, but imagine Dynamic LoRA, multimodal support, quantization, and you never leaving Chrome!

This is a significant technical advancement, especially in Memory Optimization:

- Redesigned the model-loading code to work around WebAssembly's 4 GB memory limit.
- Implemented asynchronous loading of transformer stack layers (28 for Gemma 1.1 7B).
- Reduced peak WebAssembly memory usage to less than 1% of previous requirements.

Cross-Platform Compatibility
- Compiled the C++ codebase to WebAssembly for broad browser support.
- Utilized the WebGPU API for native GPU acceleration in browsers.

Here's why this matters:

1. Privacy: No need to send data to remote servers.
2. Cost-Efficiency: Eliminates server expenses.
3. Offline Capabilities: Use powerful AI without an internet connection.

Blog: https://research.google/blog/unlocking-7b-language-models-in-your-browser-a-deep-dive-with-google-ai-edges-mediapipe/


⚖ 𝐀𝐈 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐢𝐬 𝐂𝐨𝐩𝐲𝐫𝐢𝐠𝐡𝐭 𝐈𝐧𝐟𝐫𝐢𝐧𝐠𝐞𝐦𝐞𝐧𝐭

⚖ 𝐀𝐈 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐢𝐬 𝐂𝐨𝐩𝐲𝐫𝐢𝐠𝐡𝐭 𝐈𝐧𝐟𝐫𝐢𝐧𝐠𝐞𝐦𝐞𝐧𝐭

⚖ 𝐀𝐈 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐢𝐬 𝐂𝐨𝐩𝐲𝐫𝐢𝐠𝐡𝐭 𝐈𝐧𝐟𝐫𝐢𝐧𝐠𝐞𝐦𝐞𝐧𝐭

This bold claim is not my opinion, but it has been made in a recent "report" of a group, whose stance is recognizable in their name. It is roughly translated as "Authors' Rights Initiative". They published a report which was also presented before the EU Parliament according to the LinkedIn post below.

I am not really interested in politics, but as an EU citizen I am of course somewhat interested in a reasonable and practical version of the EU AI Act. Not saying there should not be rules around data and AI, but this report is obviously very biased towards one side.

While I think the report itself does not deserve attention, I post it in the hope that you find more examples, where they did not address the issue adequately. Feel free to add to my LinkedIn posts (where the original authors will see it) or here.

[en] Executive summary: https://urheber.info/media/pages/diskurs/ai-training-is-copyright-infringement/3b900058e6-1725460935/executive-summary_engl_final_29-08-2024.pdf
[de] Full report: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4946214

LinkedIn: https://www.linkedin.com/posts/activity-7238912869268959232-6cFx


The New York Times did a fun quiz to test your ability to detect whether a video is AI-generated or real. They put Runway, Kling, and Sora to the test.

I got 10/10 🤓 —how about you?

https://www.nytimes.com/interactive/2024/09/09/technology/ai-video-deepfake-runway-kling-quiz.html


Quick Gradio update and the app is back!!

https://huggingface.co/spaces/sourceoftruthdata/sot_autotrain_dreambooth_v1.1

Free image generation!


Interested in learning about everything Image?

With the rise of recent interest in Vision Language Models (VLMs), we decided to make a push to include an ImageField within Argilla! This means any open source developer can now work on better models for vision ML tasks too and we would like to show you how.

We would love to introduce this new feature to you, so we've prepared a set of notebooks to go over some common image scenarios.
finetune an CLIP retrieval model with sentence transformers
use ColPali+ Qwen VL for RAG and log the results to Argilla
image-generation preference: creating multi-modal preference datasets for free using Hugging Face inference endpoints.

See you on Thursday!

https://lu.ma/x7id1jqu


Hi, just a brief follow-up on our Guided Reasoning (GuiR) system:

I've created a template space that facilitates testing:

1. Duplicate space https://huggingface.co/spaces/logikon/guir-chat
2. Setup your own inference servers and provide details in config file
3. Add api keys as secrets
4. Your personal GuiR playground is ready

Cheers, Gregor


"It's Sunday night, fancy a game?"
https://kz919-can-you-beat-405b-in-chess.hf.space/
built with the one and only SN fast API:
https://sambanova.ai/fast-api?api_ref=907266


Hello all,

I'm excited to share my article introducing AISAK's new flagship model, AISAK-O (Artificially Intelligent Swiss Army Knife OPTIMUM).

Read the full details here:

https://huggingface.co/blog/mandelakori/aisak-o

Key highlights of AISAK-O include:

- 8 billion parameters and a 32k token context length
- Multimodal capabilities for processing both text and visual data
- Impressive benchmark scores, surpassing GPT-4V in some areas
- Specialized in tasks like image captioning, visual reasoning, and cohesive
   content generation
- Efficient architecture competing with larger models

We're also offering a unique beta testing opportunity with access to inference code.

For more information or partnership inquiries, please contact us at mandelakorilogan@gmail.com.

I hope you find this advancement in multimodal AI as exciting as we do!
https://huggingface.co/aisak-ai/O


An example of the application of LegalKit is the production of knowledge graphs, here is a demo Space 🔗

With the update of the French legal code data model uploaded to 🤗 and the introduction of a column dedicated to HTML text, it's now easy to extract links between different articles and produce complex graphs with just a few lines of Python.

This simplified demo highlights the ease of implementation and creative potential, and enables the generation of complete data sets, although requiring a powerful graphics card for display. The framework used for the moment is D3.js, but perhaps other solutions are possible. I'd be delighted to hear your suggestions, and look forward to hearing from the community.

Link to the 🤗 Space: https://huggingface.co/spaces/louisbrulenaudet/legalkit-knowledge-graph


🌐 Introducing Edupres.ru Presentations Dataset - https://huggingface.co/datasets/nyuuzyou/edupres

Dataset highlights:
- Metadata for 44,210 presentations from edupres.ru
- 21,941 presentations available in original format
- Multilingual content: Primarily Russian, with some Ukrainian, Belarusian, and English
- Each entry includes: URL, title, description, author, publication date, file size, and download link
- Data reflects educational presentations accessible through the Edupres.ru platform
- Licensed under Creative Commons Zero (CC0) for unrestricted use

This dataset offers a unique window into online educational resources, particularly in Russian-language contexts. It provides opportunities for analyzing presentation trends, topic distributions, and language patterns in educational materials. The dataset is particularly well-suited for tasks such as text classification and text retrieval in multilingual educational settings.


This is an absolutely mind-boggling experiment!

@GuangyuRobert (Twitter Handle) from MIT has created Project Sid, which simulates over 1,000 autonomous AI agents collaborating in a Minecraft environment, operating for extended periods without human intervention. This simulation demonstrates unprecedented levels of agent interaction, decision-making, and societal development.

Agents operate independently for hours or days, showcasing advanced decision-making algorithms and goal-oriented behavior.

The simulation produced complex, emergent phenomena, including:
- Economic systems with currency (gems) and trading
- Cultural development and religious practices
- Agents even understood bribing. Priests were moving the most gems to bribe people into following them!
- Governmental structures and democratic processes

Project Sid addresses fundamental challenges in AI research:
- Coherence: Maintaining consistent agent behavior over extended periods.
- Multi-agent Collaboration: Enabling effective communication and coordination among numerous AI entities.
- Long-term Progression: Developing agents capable of learning and evolving over time.

While Minecraft serves as the initial testbed, the underlying AI architecture is designed to be game-agnostic, suggesting potential applications in various digital environments and real-world simulations.

Imagine a policy being debated by the government and how it might affect society; Sid can simulate its impact!

Even if this remains just a game experiment, the project successfully manages 1,000+ agents simultaneously, a feat that requires robust distributed computing and efficient agent architecture.

createtheimaginable John6666 jlzhou flflow


SemanticFinder now supports WebGPU thanks to @Xenova's efforts with transformers.js v3!
Expect massive performance gains. Inferenced a whole book with 46k chunks in <5min. If your device doesn't support #WebGPU use the classic Wasm-based version:
- WebGPU: https://do-me.github.io/SemanticFinder/webgpu/
- Wasm: https://do-me.github.io/SemanticFinder/

WebGPU harnesses the full power of your hardware, no longer being restricted to just the CPU. The speedup is significant (4-60x) for all kinds of devices: consumer-grade laptops, heavy Nvidia GPU setups or Apple Silicon. Measure the difference for your device here: https://huggingface.co/spaces/Xenova/webgpu-embedding-benchmark
Chrome currently works out of the box, Firefox requires some tweaking.

WebGPU + transformers.js allows to build amazing applications and make them accessible to everyone. E.g. SemanticFinder could become a simple GUI for populating your (vector) DB of choice. See the pre-indexed community texts here: https://huggingface.co/datasets/do-me/SemanticFinder
Happy to hear your ideas!

TroglodyteDerivations osanseviero Felladrin maywell Xenova ngxson ai-everyday louisbrulenaudet adorkin


💾🧠How much VRAM will you need for training your AI model? 💾🧠
Check out this app where you convert:
Pytorch/tensorflow summary -> required VRAM
or
Parameter count -> required VRAM

Use it in: http://howmuchvram.com

And everything is open source! Ask for new functionalities or contribute in:
https://github.com/AlexBodner/How_Much_VRAM
If it's useful to you leave a star 🌟and share it to someone that will find the tool useful!
More discussion in: https://x.com/AlexBodner_/status/1832054850294812679

konradhugging Jawaher786 osanseviero wyrd-code genaihuman


Last Week in Medical AI: Top Research Papers/Models
🏅(September 1  - September 7, 2024)

Medical LLM & Other Models :
- CancerLLM: Large Language Model in Cancer Domain
- MedUnA: Vision-Language Models for Medical Image
- Foundation Model for Robotic Endoscopic Surgery
- Med-MoE: MoE for Medical Vision-Language Models
- CanvOI: Foundation Model for Oncology
- UniUSNet: Ultrasound Disease Prediction
- DHIN: Decentralized Health Intelligence Network

Medical Benchmarks and Evaluations:
- TrialBench: Clinical Trial Datasets & Benchmark
- LLMs for Medical Q&A Evaluation
- MedFuzz: Exploring Robustness Medical LLMs
- MedS-Bench: Evaluating LLMs in Clinical Tasks
- DiversityMedQA: Assessing LLM Bias in Diagnosis
- LLM Performance in Gastroenterology

LLM Digital Twins:
- Digital Twins for Rare Gynecological Tumors
- DT-GPT: Digital Twins for Patient Health Forecasting

Medical LLM Applications:
- HIPPO: Explainable AI for Pathology
- LLMs vs Humans in CBT Therapy
- ASD-Chat: LLMs for Autistic Children
- LLMs for Mental Health
- LLMs for Postoperative Risk Prediction

Frameworks and Methodologies:
- Rx Strategist: LLM-based Prescription Verification
- Medical Confidence Elicitation
- Guardrails for Medical LLMs

Check the full thread : https://x.com/OpenlifesciAI/status/1832476252260712788

Thank you for your continued support and love for this series! Stay up-to-date with weekly updates on Medical LLMs, datasets, and top research papers by following @aaditya   🤗

aaditya BayesTensor models4world f0ster hapisnake Joseph717171 osanseviero louisbrulenaudet Svngoku

aaditya BayesTensor Joseph717171 Svngoku

aaditya BayesTensor Joseph717171 genaihuman


🌐 Yet another dataset - https://huggingface.co/datasets/nyuuzyou/rule34world

Dataset highlights:
- Metadata for 580,977 image files from rule34.world
- Monolingual content: English tags and metadata
- Each entry includes: URL, image URL, filepath, tags, and like count
- Data reflects images available on the Rule34.world platform up to August/September 2024
- Licensed under Creative Commons Zero (CC0) for unrestricted use
This dataset offers a unique window into online anime and art communities, particularly those focused on adult content. It provides opportunities for analyzing tagging trends, image popularity, and content patterns in user-generated art platforms.

This dataset contains high-quality images and tags, making it a great source of data for training LoRA models.


How can I make my RAG application generate real-time responses? Up until now, I have been using Groq for fast LLM generation and the Gradio Live function. I am looking for a better solution that can help me build a real-time application without any delay. @abidlabs

https://huggingface.co/spaces/kingabzpro/Real-Time-RAG


Google's Chain-of-Thought (CoT) is one of the most effective ways to improve LLMs' reasoning.

Researchers have now developed a novel approach called Strategic Chain-of-Thought (SCoT) to enhance the reasoning capabilities of large language models even further.

🧠 SCoT uses a two-stage process within a single prompt:
- Strategy Elicitation: The model first identifies and determines an effective problem-solving strategy for the given task. This becomes the strategic knowledge that guides the reasoning process.
- Strategy Application: The model then applies the identified strategic knowledge to solve the problem and generate the final answer.

Essentially, SCoT integrates strategic knowledge to guide reasoning without relying on external knowledge sources or multiple queries.

According to the research, SCoT showed significant improvements over standard CoT across various datasets, including a 21.05% increase on the GSM8K math dataset and a 24.13% increase on the Tracking_Objects spatial reasoning task.

Changes in the Prompt Structure:
The SCoT prompt typically consists of five components:
- Role: Defines the expert role the model should assume.
- Workflow: Outlines the steps for strategy identification and application.
- Rules: Specifies guidelines for generating answers.
- Initialization: Sets up the task.
- Task Input: Provides the specific problem to solve.

Strategy Generation:
The model is prompted to generate strategic knowledge relevant to the problem domain. For example, in mathematics, it might favor elegant solutions like using arithmetic series formulas over brute-force calculations.

Guided Reasoning:
Using the elicited strategy, the model then generates a chain-of-thought reasoning path. This approach aims to produce more stable and higher-quality outputs compared to standard chain-of-thought methods.

Read the full paper: https://arxiv.org/abs/2409.03271


i just made the best 0.5b model to date (again)

its name is arco and is ready to fight any 0.5b model at arc challenge

https://huggingface.co/appvoid/arco

John6666 Tonic AtAndDev louisbrulenaudet


Reposting from twitter:

Just so you all know, I'll be on vacation for the following two weeks and away from home! I'm hoping to get on at least once a day to load up some quants, but I won't be as bleeding edge and on the ball :) feel free to shoot me a message if you see one I should make!

In the meantime if you need something bleeding edge make sure to check out @MaziyarPanahi or @bullerwins who both put out great work!

Joseph717171 syrupsweety not-lain bullerwins MaziyarPanahi MarinaraSpaghetti osanseviero Whiteshadow12 victor JoeySalmons celsowm hudzax win10

Joseph717171 John6666 AIGUYCONTENT MaziyarPanahi osanseviero neoopus


FLUX Prompt Generator Updates

- https://huggingface.co/spaces/gokaygokay/FLUX-Prompt-Generator

- There are now hundreds of new selections across diverse categories, each offering a lot of choices:

Architecture, Art, Artist, Brands, Character, Cinematic, Fashion, Feelings, Geography, Human, Interaction, Keywords, Objects, People, Photography, Plots, Poses, Scene, Science, Stuff, Time, Typography, Vehicle, Video Game

- In addition to Hugging Face, I've integrated new LLM providers: Groq, OpenAI, and Claude.

- Upgraded Vision Language Models (VLMs): We now feature Qwen2-VL, JoyCaption and Florence-2-large.

- New specialized system prompts for various styles and themes, including Happy, Simple, Poster, Only Objects, No Figure, Landscape, Fantasy.

John6666 YaTharThShaRma999 KingNish ucsahin Chief-Inspector EmilyChan victor Felladrin Nyxie7 gokaygokay


Yesterday  @mattshumer released https://huggingface.co/mattshumer/Reflection-Llama-3.1-70B, an impressive model that achieved incredible results in benchmarks like MMLU. The model was fine-tuned using Reflection-Tuning and the dataset used wasn't released, but I created a small recipe with distilabel that allows generating a dataset with a similar output format:

1. We use MagPie 🐦  in combination with https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct to generate reasoning instructions.
2. We generate a response again using https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct, but we steer the LLM to generate an specific output format using a custom system prompt. In the system prompt, we instruct the LLM that it will have first to think 💭 and have reflections that will help resolving ambiguities. After that, we instruct the LLM to generate an output based on the previous thinking

In this dataset https://huggingface.co/datasets/gabrielmbmb/distilabel-reflection-tuning you can found 5 rows that I generated with this recipe. You can also found the code of the pipeline in the file called `reflection.py`.

clem prithivMLmods KonradSzafer Svngoku John6666 den0620 osanseviero gabrielmbmb louisbrulenaudet


hemanuelly hunken Sri-Vigneshwar-DJ Tonioesparza den0620 Akash3104


"LLM inference at scale with TGI". Cool blogpost: https://www.adyen.com/knowledge-hub/llm-inference-at-scale-with-tgi

Well done
@martinigoyanes @rafa-hernandez @Vidusharma @frisokingma @hannahwright @jeanmarcs @antonioramos & the whole https://huggingface.co/adyen team. Could be useful to cross-post here: https://huggingface.co/blog/community

vilarin KingNish victor prithivMLmods nbroad


💾🧠How much VRAM will you need for training your AI model? 💾🧠

Pytorch/tensorflow summary -> required VRAM

And everything is open source! Ask for new functionalities or contribute in:

If it's useful to you leave a star 🌟and share it to someone that will find the tool useful!


 I've been working on a  Space to make it super easy to create notebooks and help users quickly understand and manipulate their data!
With just a few clicks automatically generate notebooks for:

📊 Exploratory Data Analysis
🧠 Text Embeddings
🤖 Retrieval-Augmented Generation (RAG)

✨ Automatic training is coming soon!
Check it out here https://huggingface.co/spaces/asoria/auto-notebook-creator
Appreciate any feedback to improve this tool 🤗


🌟 Argilla v2.1.0 goes multi-modal: Image Field, Dark Mode, Enhanched Hugging Face Hub imports and more!

🖼 Image Field: Seamlessly work with multimodal datasets
🌓 Dark Mode: Reduce eye strain with our sleek new look
🤗 Enhanced Hugging Face Hub import with the SDK
🇪🇸 Spanish UI: Breaking language barriers

Plus more improvements to supercharge your model curation workflow!

Check out the full announcement for details and code examples: https://github.com/argilla-io/argilla/compare/v2.0.1...v2.1.0

davanstrien gabrielmbmb dvilasuero Ameeeee louisbrulenaudet clem John6666 KingNish AtAndDev

gabrielmbmb dvilasuero Ameeeee clem AtAndDev

Ameeeee clem KingNish AtAndDev gabrielmbmb


Wanted to train a FLUX model using out-of-copyright images, so I curated concept art images from NASA.

Model: https://huggingface.co/davanstrien/nasa_concept_art
Dataset: https://huggingface.co/datasets/davanstrien/nasa_concept_art

So far, training was done without captions, but I'm experimenting with using VLLMs to generate captions to see if that improves the model.


🤯 𝗔 𝗻𝗲𝘄 𝟳𝟬𝗕 𝗼𝗽𝗲𝗻-𝘄𝗲𝗶𝗴𝗵𝘁𝘀 𝗟𝗟𝗠 𝗯𝗲𝗮𝘁𝘀 𝗖𝗹𝗮𝘂𝗱𝗲-𝟯.𝟱-𝗦𝗼𝗻𝗻𝗲𝘁 𝗮𝗻𝗱 𝗚𝗣𝗧-𝟰𝗼!

@mattshumer, CEO from Hyperwrite AI, had an idea he wanted to try out: why not fine-tune LLMs to always output their thoughts in specific parts, delineated by <thinking> tags?

Even better: inside of that, you could nest other sections, to reflect critically on previous output. Let’s name this part <reflection>. Planning is also put in a separate step.

He named the method “Reflection tuning” and set out to fine-tune a Llama-3.1-70B with it.

Well it turns out, it works mind-boggingly well!

🤯 Reflection-70B beats GPT-4o, Sonnet-3.5, and even the much bigger Llama-3.1-405B!

𝗧𝗟;𝗗𝗥
🥊 This new 70B open-weights model beats GPT-4o, Claude Sonnet, et al.
⏰ 405B in training, coming soon
📚 Report coming next week
⚙ Uses GlaiveAI synthetic data
🤗 Available on HF!

I’m starting an Inference Endpoint right now for this model to give it a spin!

Check it out 👉 https://huggingface.co/mattshumer/Reflection-Llama-3.1-70B

TahirC Yuuru iandeanschaefer trollek Joseph717171


I'm building an AI for healthcare support for professionals, any advice? I could create a new app here but it needs a lot of trainer (Im newbie in this kind of stuff)
Thank you, guys!!!!


Good lord... Spent almost a day debugging this and it turns out it was an issue of gradio update incompatible with the new fastapi.
https://discuss.huggingface.co/t/huggingface-space-failed-after-working-initially/105514/8

Finally got it back online! Come chat with your favorite anime characters here:
https://huggingface.co/spaces/kz919/Persona-AI


Good folks at Epoch AI have just released their most comprehensive database yet, tracking over 800 state-of-the-art and historically notable AI models. This incredible resource provides key insights into the factors driving machine learning progress.

Since 2010, the training compute used to create AI models has been growing at a staggering rate of 4.1x per year. That means the computational power behind these models is doubling roughly every six months! And it's not just the compute that's increasing - the costs are too. Training compute costs for the largest models are doubling every nine months, with the most advanced models now costing hundreds of millions of dollars.

Interestingly, training compute has scaled up faster for language models compared to vision. While the largest vision and language models had similar compute requirements before 2020, language models have since rapidly outpaced vision models, driven by the success of transformer architectures. The size of datasets used to train language models is also doubling approximately every eight months.

Another fascinating trend is that the length of time spent training notable models is growing by about 1.2x per year. While longer training times could ease hardware constraints, there is a tradeoff to consider. For very long runs, waiting for algorithmic and hardware improvements might be more beneficial than simply extending training.

If this continues, by 2028, we will reach cluster prices in the 100 billion dollars, using 10GW of power!

Link: https://epochai.org/data/notable-ai-models


Decided to try to check how many weights in a 70b F32 model would be squashed when converted to F16 (spoiler, it's shockingly few)

The reason for this comparison is that it should represent the same percentage of squishing as bf16 to fp16

Had claude make me a script, using the new Reflection-70B, and these are the results:

Total weights: 70553706496
Fully representable: 70530215524
Squashed: 23490972
Percentage squashed: 0.03%

0.03%!!!!

A couple things to note, this uses a roundtrip of F32 -> F16 -> F32 and then torch.isclose to account for rounding errors that come up by the very nature of extremely accurate numbers, but it uses VERY small tolerances (rtol=1e-5, atol=1e-8)

This is also examining EVERY weight that was stored at F32, and for most layers I was somewhere between 0% and 0.03% of weights being squashed, no major outliers.

Overall, I feel even safer converting to F16 for llama.cpp, the extremely small number of weights that fall outside the range are likely so small that they don't actually play a role in the final output of the model at inference anyways.


@ehartford https://huggingface.co/CohereForAI/c4ai-command-r-plus dolphin when?

John6666 mahiatlinux danielus Locutusque den0620 AtAndDev


The `timm` leaderboard https://huggingface.co/spaces/timm/leaderboard has been updated with the ability to select different hardware benchmark sets: RTX4090, RTX3090, two different CPUs along with some NCHW / NHWC layout and torch.compile (dynamo) variations.

Also worth pointing out, there are three rather newish 'test' models that you'll see at the top of any samples/sec comparison:
* test_vit (https://huggingface.co/timm/test_vit.r160_in1k)
* test_efficientnet (https://huggingface.co/timm/test_efficientnet.r160_in1k)
* test_byobnet (https://huggingface.co/timm/test_byobnet.r160_in1k, a mix of resnet, darknet, effnet/regnet like blocks)

They are < 0.5M params, insanely fast and originally intended for unit testing w/ real weights. They have awful ImageNet top-1, it's rare to have anyone bother to train a model this small on ImageNet (the classifier is roughly 30-70% of the param count!). However, they are FAST on very limited hadware and you can fine-tune them well on small data. Could be the model you're looking for?


I have put together a notebook on Multimodal RAG, where we do not process the documents with hefty pipelines but natively use:
- https://huggingface.co/vidore/colpali for retrieval 📖 it doesn't need indexing with image-text pairs but just images!
- https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct for generation 💬 directly feed images as is to a vision language model with no processing to text!
I used ColPali implementation of the new 🐭 Byaldi library by @bclavie  🤗
https://github.com/answerdotai/byaldi
Link to notebook: https://github.com/merveenoyan/smol-vision/blob/main/ColPali_%2B_Qwen2_VL.ipynb

rwightman umair894 clem louisbrulenaudet s3nh John6666 BasitMustafa Johnyquest7 phanhoang resbyte denizaybey allandclive fdaudens vilarin ak0601 rreed-pha xi0v Rajaram1996 Rayvee oceansweep parjun byteprobe Filippo

hitchhiker3010 Csplk sasikiran fsommers rogermt navin7 sambarnett96 oceansweep ysdede shreyamondal

rreed-pha oceansweep Yassmen madstuntman11


How do i access llama 3.1 70b in my space ?

this doesn't seem to work, can someone help me with a working code

from transformers import AutoConfig

config = AutoConfig.from_pretrained("meta-llama/Meta-Llama-3.1-70B", revision="main")
config.rope_scaling = {"type": "llama3", "factor": 8.0}

model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-70B", config=config, use_auth_token=True)


🚀 𝗪𝗵𝗲𝗿𝗲 𝘀𝗰𝗮𝗹𝗶𝗻𝗴 𝗹𝗮𝘄𝘀 𝗮𝗿𝗲 𝘁𝗮𝗸𝗶𝗻𝗴 𝘂𝘀 : 𝗯𝘆 𝟮𝟬𝟮𝟴, 𝗔𝗜 𝗖𝗹𝘂𝘀𝘁𝗲𝗿𝘀 𝘄𝗶𝗹𝗹 𝗿𝗲𝗮𝗰𝗵 𝘁𝗵𝗲 𝗽𝗼𝘄𝗲𝗿 𝗰𝗼𝗻𝘀𝘂𝗺𝗽𝘁𝗶𝗼𝗻 𝗼𝗳 𝗲𝗻𝘁𝗶𝗿𝗲 𝗰𝗼𝘂𝗻𝘁𝗿𝗶𝗲𝘀

Reminder : “Scaling laws” are empirical laws saying that  if you keep multiplying your compute by x10, your models will mechanically keep getting better and better.

To give you an idea, GPT-3 can barely write sentences, and GPT-4, which only used x15 its amount of compute, already sounds much smarter than some of my friends (although it's not really - or at least I haven't tested them side-by side). So you can imagine how far a x100 over GPT-4 can take us.

🏎 As a result, tech titans are racing to build the biggest models, and for this they need gigantic training clusters.

The picture below shows the growth of training compute: it is increasing at a steady exponential rate of a x10 every 2 years. So let’s take this progress a bit further:
- 2022: starting training for GPT-4 : 10^26 FLOPs, cost of $100M
- 2024: today, companies start training on much larger clusters like the “super AI cluster” of Elon Musk’s xAI, 10^27 FLOPS, $1B
- 2026 : by then clusters will require 1GW, i.e. around the full power generated by a nuclear reactor
- 2028: we reach cluster prices in the 100 billion dollars, using 10GW, more than the most powerful power stations currently in use in the US. This last size seems crazy, but Microsoft and OpenAI already are planning one.

Will AI clusters effectively reach these crazy sizes where the consume as much as entire countries?
➡ Three key ingredients of training might be a roadblock to scaling up :
💸 Money: but it’s very unlikely, given the potential market size for AGI, that investors lose interest.
⚡ Energy supply at a specific location
📚 Training data: we’re already using 15 trillion tokens for Llama-3.1 when Internet has something like 60 trillion.

🤔 I’d be curious to hear your thoughts: do you think we’ll race all the way there?


🌐 Introducing PPT Online Dataset - https://huggingface.co/datasets/nyuuzyou/pptonline

Dataset highlights:
- Metadata for 1,418,349 PowerPoint (.ppt) files from ppt-online.org
- Multilingual content: Russian, Ukrainian, Belarusian, Kazakh, English, and others
- Each entry includes: Unique ID, title, category, download link, file size, and content snippet
- Data reflects presentations accessible through the PPT Online platform
- Licensed under Creative Commons Zero (CC0) for unrestricted use

This dataset offers a unique window into online educational resources, particularly in Eastern European and Central Asian contexts. It provides opportunities for analyzing presentation trends, topic distributions, and language patterns in educational materials.


Hey everyone 🤗!
We (finegrain) have created some custom ComfyUI nodes to use our refiners micro-framework inside comfy! 🎉

We only support our new Box Segmenter at the moment, but we're thinking of adding more nodes since there seems to be a demand for it. We leverage the new (beta) Comfy Registry to host our nodes. They are available at: https://registry.comfy.org/publishers/finegrain/nodes/comfyui-refiners. You can install them by running:
```
comfy node registry-install comfyui-refiners
```
Or by unzipping the archive you can download by clicking "Download Latest" into your `custom_nodes` comfy folder.
We are eager to hear your feedbacks and suggestions for new nodes and how you'll use them! 🙏

limiteinductive John6666 lunarflu djuna catwell


🙋🏻♂Hey there folks,

Did you see the new coding model from @01-ai ?

collection : https://huggingface.co/collections/01-ai/yi-coder-66bdb00f5bdd611f9a008f30
demo : https://huggingface.co/spaces/Tonic/Yi-Coder-9B

achieves SOTA on benchmarks , 125K context window , 55 languages including Docker, Js and many more 🚀


If you have documents that do not only have text and you're doing retrieval or RAG (using OCR and LLMs), give it up and give ColPali and vision language models a try 🤗

Why? Documents consist of multiple modalities: layout, table, text, chart, images. Document processing pipelines often consist of multiple models and they're immensely brittle and slow. 🥲

How? ColPali is a ColBERT-like document retrieval model built on PaliGemma, it operates over image patches directly, and indexing takes far less time with more accuracy. You can use it for retrieval, and if you want to do retrieval augmented generation, find the closest document, and do not process it, give it directly to a VLM like Qwen2-VL (as image input) and give your text query. 🤝

This is much faster + you do not lose out on any information + much easier to maintain too! 🥳

Multimodal RAG https://huggingface.co/collections/merve/multimodal-rag-66d97602e781122aae0a5139 💬
Document AI (made it way before, for folks who want structured input/output and can fine-tune a model) https://huggingface.co/collections/merve/awesome-document-ai-65ef1cdc2e97ef9cc85c898e 📖

adorkin web3builder John6666 Percifal jrmasiero rwightman seek007 abishekcodes zliu AI4Industry louisbrulenaudet byteprobe muhtasham rumbleFTW

umair894 abishekcodes fsommers jithinrocs rumbleFTW


🔥 Dataset Viber 0.3 launches with Synthesizer to synthesise data with a human in the loop, for free, using open source models with Argilla's distilabel but within a quick-and-easy Gradio Interface.

Why? Not trying to be all fancy and formal just to iterate on your data and to get familiar with your prompts and the produced data. Under the hood, it relies on Hugging Face Inference endpoints and the latest LLMs and VLMs like Meta Llama 3.1 and BlackForest Labs Flux models.

An addition to the other Interfaces that are already support.
- CollectorInterface: Lazily collect data of model interactions without human annotation.
- AnnotatorInterface: Walk through your data and annotate it with models in the loop.
- Synthesizer: Synthesize data with distilabel in the loop.
- BulkInterface: Explore your data distribution and annotate in bulk.

⭐ Give some good vibes: https://github.com/davidberenstein1957/dataset-viber


Datapluck: Portability Tool for Huggingface Datasets

"I found myself recently whipping up notebooks just to pull huggingface datasets locally, annotate or operate changes and update them again. This happened often enough that I made a cli tool out of it, which I've been using successfully for the last few months.

While huggingface uses open formats, I found the official toolchain relatively low-level and not adapted to quick operations such as what I am doing."
~ @omarkamali

Link : https://omarkama.li/blog/datapluck


Just wrapped up a deep dive into the latest lecture on building LLMs, such as ChatGPT, from @Stanford  CS229 course. Here are my top takeaways:

🔍 Understanding the Components: LLMs like ChatGPT, Claude, and others are more than just neural networks; they are a complex blend of architecture, training loss, data evaluation, and systems. Knowing how these components work together is key to improving and scaling these models.

📊 Scaling Matters: Performance improves predictably with more data, bigger models, and greater computational power. However, balancing these factors is crucial to avoid overfitting and resource waste.

📈 Data is King: LLMs are trained on trillions of tokens scraped from the internet, but the quality of this data matters immensely. Rigorous filtering and deduplication processes are essential to maintaining data integrity.

🏗 Pre-Training vs. Post-Training: While pre-training equips the model with general knowledge, post-training (like RLHF) fine-tunes it to follow human-like responses, reducing toxic outputs and improving alignment with human values.

🌐 Reinforcement Learning from Human Feedback (RLHF): This technique allows LLMs to maximize outputs that align with human preferences, making models more reliable and accurate.

💡 Why It Matters: Understanding these processes not only helps us appreciate the complexity behind our everyday AI tools but also highlights the challenges and opportunities in the ever-evolving field of AI.

Whether you’re in tech, data science, or just AI-curious, staying updated on these advancements is crucial. LLMs are not just transforming industries; they’re redefining the future of human-computer interaction!

I just realized this was almost 2 hours long...

Link: https://www.youtube.com/watch?v=9vM4p9NN0Ts


I just bought HF Pro but i don't know how many request per month i can get, if i request 1 time every 5s, around 2k token, is the pro account enough?, thanks for reading


My tool calling playgrounds repo has been updated again to include the use of flux1-schnell or dev image generation. This functionality is similar to using Dall-E 3 via the `@` decorator in ChatGPT. Once the function is selected, the model will either extract or improve your prompt (depending on how you ask).

I have also included 2 notebooks that cover different ways to access Flux for your specific use case. The first method covers how to access flux via LitServe from Lightning AI. LitServe is a bare-bones inference engine with a focus on modularity rather than raw performance. LitServe supports text generation models as well as image generation, which is great for some use cases, but does not provide the caching mechanisms from a dedicated image generation solution.

Since dedicated caching mechanisms are so crucial to performance, I also included an example for how to integrate SwarmUI/ComfyUI to utilize a more dedicated infrastructure that may already be running as part of your tech stack. Resulting in a Llama-3.1 capable of utilizing specific ComfyUI JSON configs, and many different settings.

Lastly, I tested the response times for each over a small batch request to simulate a speed test.

It becomes clear quickly how efficient caching mechanisms can greatly reduce the generation time, even in a scenario where another model is called. An average 4.5 second response time is not bad at all when you consider that an 8B model is calling a 12B parameter model for a secondary generation.

Repo: https://github.com/tdolan21/tool-calling-playground
LitServe: https://github.com/Lightning-AI/LitServe
SwarmUI: https://github.com/mcmonkeyprojects/SwarmUI


🥳  𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀 𝗔𝗴𝗲𝗻𝘁𝘀 𝗻𝗼𝘄 𝘀𝘂𝗽𝗽𝗼𝗿𝘁𝘀 𝗠𝘂𝗹𝘁𝗶-𝗮𝗴𝗲𝗻𝘁 𝘀𝘆𝘀𝘁𝗲𝗺𝘀!

Multi-agent systems have been introduced in Microsoft's framework Autogen. It simply means having several agents working together to solve your task instead of only one : this paradigm empirically yields better performance on most benchmarks. The reason for this better performance is conceptually simple: for many tasks, rather than using a do-it-all system, you would prefer to specialize units on sub-tasks. Here, having agents with separate tool sets and memories allows to achieve efficient specialization.

You can now easily build hierarchical multi-agent systems with transformers.agents (not released yet, use the dev version)

To do so, encapsulate the agent in a ManagedAgent object. This object needs arguments agent, name, and a description, which will then be embedded in the manager agent's system prompt to let it know how to call this managed agent, as we also do for tools.

Cf the example in the image! We'll keep building on this paradigm in the upcoming weeks 🚀

Read more in the doc 👉 https://github.com/huggingface/transformers/blob/main/docs/source/en/agents_advanced.md

Checkout an advanced multi-agent system that tops the GAIA leaderboard 👉 https://github.com/aymeric-roucher/GAIA/blob/main/gaia_multiagent.py

ibrahim313 John6666 osanseviero Kaoeiri dsmonk Csplk KingNish whitebill Winnougan


the new version of Enigma, our code-instruct specialist, is out now:
- https://huggingface.co/ValiantLabs/Llama3.1-8B-Enigma is trained on code-instruct and general chat data.
- the updated code-instruct database is available now as well: https://huggingface.co/datasets/sequelbox/Tachibana

more to come soon!


🐣Ai2 Releasing OLMoE!
OLMoE-1B-7B-Instruct is a Mixture-of-Experts LLM with 1B active and 7B total parameters, and, OLMoE is 100% open-source in model, code-base, datasets!

🦖Paper: https://arxiv.org/abs/2409.02060

🤗Model: https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct

💾Datasets: https://huggingface.co/datasets/allenai/OLMoE-mix-0924


If you want a clear understanding of the environmental impacts of AI throughout its entire lifecycle, this primer by @sasha @brunatrevelin and @meg is a must-read.

It brilliantly explains which types of impacts occur, when they happen, and why they matter.

https://huggingface.co/blog/sasha/ai-environment-primer

brunatrevelin John6666 not-lain BrigitteTousi louisbrulenaudet


The new Qwen-2 VL models seem to perform quite well in object detection. You can prompt them to respond with bounding boxes in a reference frame of 1k x 1k pixels and scale those boxes to the original image size.

You can try it out with my space https://huggingface.co/spaces/maxiw/Qwen2-VL-Detection

de-Rodrigo dsmonk tosouth hxypqr mrdbourke SvPolina akazakov Panerlu iiBLACKii


The MERIT Dataset: Modelling and Efficiently Rendering Interpretable
  Transcripts (2409.00447)

A few weeks ago, we uploaded the MERIT Dataset 🎒📃🏆 into Hugging Face 🤗!

Now, we are excited to share the Merit Dataset paper via arXiv! 📃💫
https://huggingface.co/papers/2409.00447

The MERIT Dataset is a fully synthetic, labeled dataset created for training and benchmarking LLMs on Visually Rich Document Understanding tasks. It is also designed to help detect biases and improve interpretability in LLMs, where we are actively working. 🔧🔨

MERIT contains synthetically rendered students' transcripts of records from different schools in English and Spanish. We plan to expand the dataset into different contexts (synth medical/insurance documents, synth IDS, etc.) Want to collaborate? Do you have any feedback? 🧐

Resources:

- Dataset: https://huggingface.co/datasets/de-Rodrigo/merit
- Code and generation pipeline: https://github.com/nachoDRT/MERIT-Dataset

PD: We are grateful to Hugging Face 🤗 for providing the fantastic tools and resources we find in the platform and, more specifically, to @nielsr for sharing the fine-tuning/inference scripts we have used in our benchmark.


@victor Sorry for the repetitiveness.

I'm not sure if Post is the right place to report such an error, but it seems to be a server error unrelated to the Zero GPU space error the other day, so I don't know where else to report it.

Since this morning, I have been getting a strange error when running inference from space in Gradio 3.x.
Yntec (https://huggingface.co/Yntec) discovered it, but he is not in the Pro subscription, so I am reporting it on behalf of him.

The error message is as follows: 1girl and other prompts will show cached output, so experiment with unusual prompts.

Thank you in advance.

https://huggingface.co/spaces/John6666/blitz_diffusion_error
https://huggingface.co/spaces/John6666/GPU-stresser-t2i-error
```
ValueError: Could not complete request to HuggingFace API, Status Code: 500, Error: unknown error, Warnings: ['CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 14.75 GiB total capacity; 1.90 GiB already allocated; 3.06 MiB free; 1.95 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF', 'There was an inference error: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 14.75 GiB total capacity; 1.90 GiB already allocated; 3.06 MiB free; 1.95 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF']
```


I am integrating Azure Cosmos DB, the database system that backs GPT conversations into my workflow, and experimenting with new patterns to accelerate dataset evolution for evaluation and training of AI.

While initially using it for research prompts and research outputs using my GPT-4o client here which can interface and search ArXiv, I am excited to try out some new features specifically for AI at scale.  Research on memory augmentation is shown.  https://huggingface.co/spaces/awacke1/GPT-4o-omni-text-audio-image-video

  https://huggingface.co/spaces/awacke1/AzureCosmosDBUI


🚀 Introducing Hugging Face's Multilingual Speech-to-Speech! 🎤
💬Our modular, cross-platform pipeline to run GPT4o-like experiences on device can now seamlessly switch languages mid-conversation with an imperceptible 100ms delay.

🌟 Building on an amazing early reception with 2600 stars on GitHub 🌟
🚀 We are expanding the library to support multiple languages
🔥 Try it out with a flag: --language fr
🤯 Or don't set the flag and let the system detect the language

💡 What feature should we add next?

prithivMLmods osanseviero John6666 THEFIG


🧭 Guided Reasoning

👋Hi everyone,

We've been releasing Guided Reasoning:

Our AI guides walk your favorite LLM through complex reasoning problems.

🎯 Goals:

1⃣ Reliability. AIs consistently follow reasoning methods.
2⃣ Self-explainability. AIs see reasoning protocols and can explain internal deliberation.
3⃣ Contestability. Users may amend AI reasoning and revise plausibility assessments.

Try out Guided Reasoning with our light demo chatbot, powered by 🤗 HuggingFace's free Inference Api and small LLMs. (Sorry for poor latency and limited availability -- we are currently searching for 💸 compute sponsors to run more powerful models, faster, and optimize guided reasoning performance.)

Built on top of Logikon's open-source AI reasoning analytics.

Demo chat app: https://huggingface.co/spaces/logikon/benjamin-chat
Github: https://github.com/logikon-ai/logikon
Technical report: https://arxiv.org/abs/2408.16331

➡ Check it out and get involved! Looking forward to hearing from you.


 Fine-tuned Phi-3.5 Chatbot

This project presents a fine-tuned version of Microsoft's Phi-3.5 model, optimized for enhanced conversational abilities and general knowledge tasks.

Model Details
- Base model: microsoft/Phi-3.5-mini-instruct
- Fine-tuning method: PEFT (Parameter-Efficient Fine-Tuning)
- Training data: [Brief description of your dataset]

 Features
- Improved response generation for a wide range of topics
- Enhanced context understanding and coherence
- Optimized for deployment on Hugging Face Spaces

Usage
This model can be used for various natural language processing tasks, including:
- General conversation
- Question answering
- Task instructions
- Creative writing

Try out the model here :
  https://huggingface.co/spaces/sagar007/phi3.5_mini_instruct_finetune

Limitations
While this fine-tuned model shows improved performance, users should be aware of potential biases and limitations inherent in language models. Always critically evaluate the model's outputs.

 Feedback
I welcome any feedback, suggestions, or questions about this project. Feel free to open an issue or contribute to further improvements!

#Phi35 #FineTuning #NLP #MachineLearning #HuggingFace


Hyperfast Contextual Custom LLM with Agents, Multitokens, Explainable AI, and Distillation https://mltblog.com/4dNPSnB

New additions to this ground-breaking system include multi-token distillation when processing prompts, agents to meet user intent, more NLP, and a command prompt menu accepting both standard prompts and various actions.

I also added several illustrations, featuring xLLM in action with a full session and sample commands to fine-tune in real-time. All the code, input sources (anonymized corporate corpus from fortune 100 company), contextual backend tables including embeddings, are on GitHub. My system has zero weight, no transformer, and no neural network. It relies on explainable AI, does not require training, is fully reproducible, and fits in memory. Yet your prompts can retrieve relevant full text entities from the corpus with no latency — including URLs, categories, titles, email addresses, and so on — thanks to well-designed architecture.

Read more, get the code, paper and everything for free, at https://mltblog.com/4dNPSnB