Korvus: Single-query RAG with Postgres

Hey folks!

We built Korvus, an open-source RAG (Retrieval-Augmented Generation) pipeline that consolidates the entire RAG workflow - from embedding generation to text generation - into a single SQL query, significantly reducing architectural complexity and latency.

https://github.com/postgresml/korvus

Here's some of the highlights:

- Full RAG pipeline (embedding generation, vector search, reranking, and text generation) in one SQL query
- SDKs for Python, JavaScript, and Rust (more languages planned)
- Built on PostgreSQL, leveraging pgvector and pgml
- Open-source, with support for open models
- Designed for high performance and scalability

We're eager to get feedback from the community and welcome contributions. Check out our GitHub repo for more details:

https://github.com/postgresml/korvus


I am working to add automatic speech bubbles to the AI Comic Factory!

This is not finished yet - there are a few things to tweak - but so far results are pretty promising!

(it's available through a "beta" toggle)

prithivMLmods MattBee Norod78 fdaudens sirjamesgray o0Rodney0o Tkap0176 victor KingNish Tusharkn12 osanseviero John6666 nmdeal louisbrulenaudet Ajayk Ramikan-BR yazdaabdul Sohaibsoussi imangshu MaziyarPanahi SouthbayJay fifteen42 Pent

TNVMike Tkap0176 tusharsoni00786 Ramikan-BR Cassoni alandarkx UsmanComics HemiHDR

digiplay Ramikan-BR MaziyarPanahi MAGMA91 dongnt


Continuing the series of datasets with code from Russian platforms: GitFlic Code Dataset - https://huggingface.co/datasets/nyuuzyou/gitflic-code.
📊 Dataset highlights:

60 GB of unique code extracted from over 967 GB of analyzed data
12,527 repositories
5,983,358 unique code files
692 different file types
Multilingual: various programming languages

 🌐 Sourced from GitFlic, the first Russian service for storing and working with source code, based on the Git version control system.


Vision language models are blind (2407.06581)

Researchers from Auburn University and the University of Alberta have explored the limitations of Vision Language Models (VLMs) in their recently published paper titled "Vision language models are blind." (https://huggingface.co/papers/2407.06581)

Key Findings:🔍
VLMs, including GPT-4o, Gemini-1.5 Pro, Claude-3 Sonnet, and Claude-3.5 Sonnet, struggle with basic visual tasks.
Tasks such as identifying where lines intersect or counting basic shapes are challenging for these models.
The authors noted, "The shockingly poor performance of four state-of-the-art VLMs suggests their vision is, at best, like of a person with myopia seeing fine details as blurry, and at worst, like an intelligent person that is blind making educated guesses"(Vision Language Models Are Blind; 2024).

Human-like Myopia? 👓
VLMs may have a blind spot similar to human myopia.
This limitation makes it difficult for VLMs to perceive details.
Suggests a potential parallel between human and machine vision limitations.

Technical Details: 🔧
The researchers created a new benchmark called BlindTest.
BlindTest consists of simple visual tasks to evaluate VLMs low-level vision capabilities.
Four VLMs were assessed using BlindTest.
Many shortcomings were revealed in the models ability to process basic visual information.

Learn More: 🖼
For a deeper dive into this research, check out the project page: https://vlmsareblind.github.io/


Introducing Whisper Timestamped: Multilingual speech recognition with word-level timestamps, running 100% locally in your browser thanks to 🤗 Transformers.js! Check it out!
👉 https://huggingface.co/spaces/Xenova/whisper-word-level-timestamps 👈

This unlocks a world of possibilities for in-browser video editing! 🤯 What will you build? 😍

Source code: https://github.com/xenova/transformers.js/tree/v3/examples/whisper-word-timestamps

BoscoTheDog fdaudens boapps Ramikan-BR Noahka bwrasa GPT007 osanseviero John6666 ralphilius louisbrulenaudet GordonM fedyano


Ghost 8B Beta is a large language model developed with goals that include excellent multilingual support, superior knowledge capabilities, and cost-effectiveness. The model comes in two context length versions, 8k and 128k, along with multilingual function tools support by default.
* The languages supported are 🇺🇸 English, 🇫🇷 French, 🇮🇹 Italian, 🇪🇸 Spanish, 🇵🇹 Portuguese, 🇩🇪 German, 🇻🇳 Vietnamese, 🇰🇷 Korean and 🇨🇳 Chinese.
* 👨💻 Try on Spaces: https://huggingface.co/spaces/lamhieu/ghost-8b-beta-8k
* 📋 Official website: https://ghost-x.org/docs/models/ghost-8b-beta


𝐍𝐞𝐰 𝐝𝐞𝐜𝐨𝐝𝐢𝐧𝐠 𝐭𝐞𝐜𝐡𝐧𝐢𝐪𝐮𝐞 𝐢𝐧 𝐭𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫𝐬 𝐬𝐢𝐠𝐧𝐢𝐟𝐢𝐜𝐚𝐧𝐭𝐥𝐲 𝐫𝐞𝐝𝐮𝐜𝐞𝐬 𝐡𝐚𝐥𝐥𝐮𝐜𝐢𝐧𝐚𝐭𝐢𝐨𝐧𝐬 👏

DoLa decoding, which made a conference paper at ICLR '24, has just been merged in Transformers by @joaogante and Yung-Sung Chuang.
This new decoding method is simple yet extremely impressive!

Reminder: Decoder LLMs (the GPT kind of LLM, the most common one) generate their outputs one token at a time: at each step, given a current text, they compute a logit for each token in their vocabulary that should represent the probability of this token coming next.

Then they either pick the highest logit token (greedy decoding) or sample one with a probability defined by the logits (sampling).

The authors of DoLa wanted to improve that simple method.

They knew this established fact that transformer LMs encode low-level info (like base syntax) in early layers and more high-level info like knowledge in the later layers.

💡 This gave them their key idea: During decoding, rather than picking the token with the highest logit, 𝘄𝗵𝘆 𝗻𝗼𝘁 𝗽𝗶𝗰𝗸 𝘁𝗵𝗲 𝘁𝗼𝗸𝗲𝗻 𝘄𝗶𝘁𝗵 𝘁𝗵𝗲 𝗺𝗼𝘀𝘁 𝗶𝗺𝗽𝗿𝗲𝘀𝘀𝗶𝘃𝗲 𝗶𝗻𝗰𝗿𝗲𝗮𝘀𝗲 𝗶𝗻 𝗹𝗼𝗴𝗶𝘁 𝗮𝗰𝗿𝗼𝘀𝘀 𝗹𝗮𝘆𝗲𝗿𝘀?

This gives impressive results:
🚀 𝟱% - 𝟮𝟬% 𝗯𝗮𝘀𝗲 𝗽𝗼𝗶𝗻𝘁𝘀 𝗶𝗻𝗰𝗿𝗲𝗮𝘀𝗲 𝗮𝗰𝗿𝗼𝘀𝘀 𝘁𝗵𝗲 𝗯𝗲𝗻𝗰𝗵𝗺𝗮𝗿𝗸𝘀
🚀 For instance on TruthfulQA / Open-ended, across all model sizes the increase in truthfulness is 14 base points, which is 𝗮𝗿𝗼𝘂𝗻𝗱 𝟰𝟬% 𝗶𝗺𝗽𝗿𝗼𝘃𝗲𝗺𝗲𝗻𝘁 𝗰𝗼𝗺𝗽𝗮𝗿𝗲𝗱 𝘁𝗼 𝘀𝘁𝗮𝗻𝗱𝗮𝗿𝗱 𝗱𝗲𝗰𝗼𝗱𝗶𝗻𝗴!

🤔 Wouldn't decoding take longer because of this added contrasting step? 👉 𝗧𝗵𝗲 𝗿𝘂𝗻𝘁𝗶𝗺𝗲 𝗶𝗻𝗰𝗿𝗲𝗮𝘀𝗲 𝗶𝘀 𝗻𝗲𝗴𝗹𝗶𝗴𝗶𝗯𝗹𝗲, 𝟭 𝘁𝗼 𝟴% 𝗼𝗻𝗹𝘆.

Paper added to my collection 👉 https://huggingface.co/collections/m-ric/optimization-mechanics-661d543a5fc6ca1dc84284a0

joaogante kerry0202 Maxwell-Jia ogozcelik alemiaschi patf82 osanseviero jukofyork ddh0


Forget any document retrievers, use ColPali 💥💥

Document retrieval is done through OCR + layout detection, but you are losing a lot of information in between, stop doing that! 🤓

ColPali uses a vision language model, which is better in doc understanding 📑
ColPali: https://huggingface.co/vidore/colpali (mit license!)
Blog post: https://huggingface.co/blog/manu/colpali
The authors also released a new benchmark for document retrieval:
ViDoRe Benchmark: https://huggingface.co/collections/vidore/vidore-benchmark-667173f98e70a1c0fa4db00d
ViDoRe Leaderboard: https://huggingface.co/spaces/vidore/vidore-leaderboard

ColPali marries the idea of modern vision language models with retrieval 🤝

The authors apply contrastive fine-tuning to SigLIP on documents, and pool the outputs (they call it BiSigLip). Then they feed the patch embedding outputs to PaliGemma and create BiPali 🖇
BiPali natively supports image patch embeddings to an LLM, which enables leveraging the ColBERT-like late interaction computations between text tokens and image patches (hence the name ColPali!) 🤩

The authors created the ViDoRe benchmark by collecting PDF documents and generate queries from Claude-3 Sonnet.
ColPali seems to be the most performant model on ViDoRe. Not only this, but is way faster than traditional PDF parsers too!

ucsahin louisbrulenaudet Jebadiah diasbalmash osanseviero Zhofang


LivePortrait: Efficient Portrait Animation with Stitching and
  Retargeting Control (2407.03168)

🔥🎭🌟 New Research Alert - LivePortrait (Avatars Collection)! 🌟🎭🔥
📄 Title: LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control 🔝

📝 Description: LivePortrait is an efficient video-driven portrait animation framework that uses implicit keypoints and stitching/retargeting modules to generate high-quality, controllable animations from a single source image.

👥 Authors: @cleardusk, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang

🤗 Demo: https://huggingface.co/spaces/KwaiVGI/LivePortrait

📄 Paper: https://huggingface.co/papers/2407.03168

🌐 Github Page: https://liveportrait.github.io/
📁 Repository: https://github.com/KwaiVGI/LivePortrait

🔥 Model 🤖: https://huggingface.co/KwaiVGI/LivePortrait

🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers

🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers

🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #LivePortrait #PortraitAnimation #ComputerVision #MachineLearning #DeepLearning #ComputerGraphics #FacialAnimation #GenerativeAI #RealTimeRendering #AI

DmitryRyumin Andzej-75 bobros dessignAI cleardusk jimdigitart GPT007 Blucky aoezdTchibo victor

DmitryRyumin Shinku Andzej-75 cleardusk osanseviero


Hello.

I'm excited to announce that 'LivePortrait' has been released, and we've developed an advanced application service using it. We've made this service available for free on our Discord server (https://discord.gg/openfreeai). To achieve this, we've deployed four Nvidia H100 GPUs and optimized our code for multi-GPU processing.

On our Discord server, we've named this channel "Mimic Face." This incredible free service allows you to create an interactive video in just a few seconds based on your photo. Simply upload a photo and select a face video you want to mimic. The AI will then replicate your face, mimicking gestures, expressions, mouth movements, eye movements, and even blinking naturally.

The process is extremely simple, and the results are impressively realistic. Experience this fantastic service for yourself, and stay tuned for more exciting AI services we plan to introduce in the future.

Open Service link: https://discord.gg/openfreeai

Channel Direct Link: https://discord.com/channels/1228254992729767996/1259659402118692896

aiqtech fantaxy fantos ginipick MaziyarPanahi julien-c victor Ramikan-BR jimdigitart seawolf2357

aiqtech fantaxy fantos ginipick Alpaggos julien-c Ramikan-BR

aiqtech fantaxy Blane187 julien-c victor Ramikan-BR seawolf2357


# LivePortrait 1-Click Installers and full tutorials for Windows and Cloud (useful for Mac users) (Massed Compute, RunPod and a free Kaggle Account)

I know there are a lot of experts around here that can install and use easily. But I have prepared solid tutorials for newbies and shown how to use this amazing top quality app LivePortrait. I have to say congrats to the developers.

I am also a researcher you can see my LinkedIn profile here but recently I am shifted into AI lectures : https://www.linkedin.com/in/furkangozukara/

Both Windows and Cloud tutorial has manually written (100% accurate) captions / subtitles. Also both have manually written by me very detailed video chapters.

Windows LivePortrait Tutorial : https://youtu.be/FPtpNrmuwXk

Cloud LivePortrait Tutorial : Massed Compute, RunPod & Kaggle : https://youtu.be/wG7oPp01COg

## Windows LivePortrait Tutorial Video Chapters

- 0:00 Introduction to LivePortrait: A cutting-edge open-source application for image-to-animation conversion
- 2:20 Step-by-step guide for downloading and installing the LivePortrait Gradio application on your device
- 3:27 System requirements and installation process for LivePortrait
- 4:07 Verifying the successful installation of required components
- 5:02 Confirming installation completion and preserving installation logs
- 5:37 Initiating the LivePortrait application post-installation
....


🚨 New Release: ultralytics8.2.51
🍺Live Space : https://huggingface.co/spaces/prithivMLmods/YOLO-VIDEO , Duplicate the Space to avoid queuing issues.
🍺T4  Colab : https://colab.research.google.com/drive/1BKgFUfk2Me1cSPFmbtZSVCn_4cYImPO-?au
👉🏻For HPC, use A100/T4 under controlled conditions.
👉🏻Speed Estimation, Object Counting, Distance Calculation, Workout Monitoring, Heatmaps,etc.

Ultralytics dropped the YOLOv8 - #Ultralytics 8.2.51 🔥, YOLOv8 is designed to be fast, accurate, and easy to use, making it an excellent choice for a wide range of object detection and tracking, instance segmentation, image classification and pose estimation tasks.

🔗 https://pypi.org/project/ultralytics/8.2.58/

🚀More Features You can try:
✅ Classes selection support added
✅ Live FPS display in the sidebar
✅ Webcam and video support added
✅ Confidence and NMS threshold option to modify.
✅ Segmentation, detection, and pose models support added.

🙀Ultralytics Live inference: https://docs.ultralytics.com/guides/streamlit-live-inference/
```
from ultralytics import solutions
solutions.inference()
### Make sure to run the file using command `streamlit run <file-name.py>`

```
⚡yolo streamlit-predict

👉🏻Advantages of Live Inference

      ☑ Seamless Real-Time Object Detection: Streamlit combined with YOLOv8 enables real-time object detection directly from your webcam feed. This allows for immediate analysis and insights, making it ideal for applications requiring instant feedback.
      ☑Efficient Resource Utilization: YOLOv8 optimized algorithm ensure high-speed processing with minimal computational resources.
🙀Ultralytics feature Models: https://docs.ultralytics.com/models/, Ultralytics new Solutions: https://docs.ultralytics.com/solutions/

👉🏻Official Documentation:
    Ultralytics YOLOv8 Documentation: Refer to the official YOLOv8 documentation for comprehensive guides and insights on various computer vision tasks and projects. 🔗 https://docs.ultralytics.com/


Check @lllyasviel Latest research for making videos of still images using hand drawing keyframes:
https://lllyasviel.github.io/pages/paints_undo/

I imported his work to gradio space here
https://huggingface.co/spaces/MohamedRashad/PaintsUndo


ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wild (2407.04172)

📢 Exciting News! Our latest paper "ChartGemma" is out! 📊

🧵1/3: ChartGemma overcomes existing chart models key limitations that rely too much on data tables. Instead, it is trained on data generated directly from chart images, capturing crucial visual trends📸🔍

🧵2/3: ChartGemma builds upon PaliGemma from Google Research and is fine-tuned on a high-quality visual instruction tuning dataset generated from Gemini Flash 1.5. 🌟📊

🧵3/3: Achieves state-of-the-art results in chart summarization, question answering, and fact-checking tasks. 🏅📊 It can also generate more accurate and realistic chart summaries. 📝🔍

Our model and data are publicly available. We also have a cool web demo. Check it out! 🚀
Demo: https://huggingface.co/spaces/ahmed-masry/ChartGemma
Code: https://github.com/vis-nlp/ChartGemma
Paper: https://huggingface.co/papers/2407.04172

megh1211 osanseviero nbroad ucsahin GPT007 John6666 damerajee AtAndDev FedericoVal holooo shivanshdhar


How's 2024 been so far? 🎉 Well, we've got the data to find out!

New dataset just dropped on the Hub: 12.5 million Reddit comments from 2024! 📊🗣

Features:
- Personal info anonymized 🕵♀
- Language detection 🌐
- Token count 🔢
- NSFW filtering 🚫

Most popular post? Classic AITA drama: "AITA for 'ruining Christmas' and being upset the only gifts I got from my family were 'joke gifts'" 🎄😅

Some things never change, huh?

Dive in: https://huggingface.co/datasets/OpenCo7/UpVoteWeb


Just updated the Journalists on 🤗 community with two new AI tools! 🚀📊

Check them out:
- Free video transcription & smart summary tool https://huggingface.co/spaces/artificialguybr/Video-Transcription-Smart-Summary
- ChartGemma - next-level chart analysis focusing on visual trends https://huggingface.co/spaces/ahmed-masry/ChartGemma

Media folks: Join our community for more tools! https://huggingface.co/JournalistsonHF


Chrome's new `window.ai` feature is going to change the web forever! 🤯 It allows you to run Gemini Nano, a powerful 3.25B parameter LLM, 100% locally in your browser!

We've also added experimental support to 🤗 Transformers.js!
- Demo: https://huggingface.co/spaces/Xenova/experimental-built-in-ai-chat
- Blog post: https://huggingface.co/blog/Xenova/run-gemini-nano-in-your-browser

victor clem GPT007 John6666 shreyask prithivMLmods indigoochoa osanseviero nbroad ucsahin MoritzLaurer eryk-mazus KwabsHug KaifResearch

clem GPT007 shreyask osanseviero louisbrulenaudet jrwren


Hi HF community! 🤗

There's a new space out in the wild! https://huggingface.co/spaces/as-cle-bert/self-reviewing-coding-assistant 🦜

It's a self-correcting and self-reviewing python coding assistant based on GPT4-o and LangChain, inspired by Codium-AI's AlphaCodium 👾

Have fun! 🐙


One more cookbook:
Agent for self-correcting Text-to-SQL 🧑💻

What if the query generated by your Text-to-SQL pipeline is correct SQL but returns wrong results?
👉 We need to add a critique step

✅ That's very simple with an agent!

Check out the notebook! 👇
https://huggingface.co/learn/cookbook/agent_text_to_sql


Is a full-featured Mac sufficient for AI and ML development compared to NVIDIA GPU systems?

Hello everyone,
We are about to make the decision to purchase a powerful Mac with maximum features for our university. This Mac will primarily serve as a development computer in the field of artificial intelligence (AI) and machine learning (ML) and will be used by a small group of users in the local network. After development, the systems will be transferred to servers that have to withstand higher loads and visitor numbers.

Planned system:
Apple Mac Studio 2023 with M2 Ultra processor
24-core CPU
76-core GPU
32-core NPU (neural engine) for machine learning
128GB RAM
1TB HDD

Our question to you:
Is a fully equipped Mac with the latest SoCs chips and integrated neural engines sufficient for the development of AI and ML systems?
Or should we rather rely on proven Windows/Linux systems with powerful NVIDIA graphics cards?

We already have several NVIDIA graphics cards available at the university:
NVIDIA Tesla T4
NVIDIA 2080Ti
NVIDIA 3080Ti

We are particularly interested in your experiences and assessments of how the performance of the Mac compares to the GPU systems mentioned.

Are there significant differences, especially in the development and training of models?

What difference would you consider to be significant?

For us, a difference of 100% or more would be considered significant.
In other words, computer A (Mac) takes twice as long to calculate as computer B (NVIDIA system).

Many thanks in advance for your answers and experience!
Best regards,

Oliver


Hi everyone,

I am excited to introduce our latest work, LLaMAX.  😁😁😁

LLaMAX is a powerful language model created specifically for multilingual scenarios. Built upon Meta's LLaMA series models, LLaMAX undergoes extensive training across more than 100 languages.

Remarkably, it enhances its multilingual capabilities without compromising its generalization ability, surpassing existing LLMs.

✨Highlights:

🎈 LLaMAX supports the 102 languages covered by Flores-101, and its performance in translating between low-resource languages far surpasses other decoder-only LLMs.

🎈 Even for languages not covered in Flores-200, LLaMAX still shows significant improvements in translation performance.

🎈 By performing simple SFT on English task data, LLaMAX demonstrates impressive multilingual transfer abilities in downstream tasks.

🎈 In our paper, we discuss effective methods for enhancing the multilingual capabilities of LLMs during the continued training phase.

We welcome you to use our model and provide feedback.

More Details:

🎉  Code:  https://github.com/CONE-MT/LLaMAX/

🎉  Model:   https://huggingface.co/LLaMAX/

yqlu LLaMAX John6666 boapps Ramikan-BR GPT007 qnguyen3 Joseph717171 htafer osanseviero paulml ruben-wleon kramp apol

Gutssu Ramikan-BR GPT007 prithivMLmods yoeldcd Joseph717171 osanseviero juancopi81 MikeMpapa

Symbol-LLM Ramikan-BR GPT007 Joseph717171 ucsahin FilipeR


Hey everyone,

We're thrilled to introduce our latest project: a hand-curated list of the best-curated lists related to artificial intelligence! 🚀

Check it out here: https://github.com/zhimin-z/awesome-awesome-artificial-intelligence


📍Excited to make public a series of checkpoints !

- Final checkpoints after self-training with ENVISIONS framework
- Cover math, logic, and agent domains
- Include 7B / 13B

📕 Check our paper:
Title: Interactive Evolution: A Neural-Symbolic Self-Training Framework For Large Language Models
Link: https://arxiv.org/abs/2406.11736


Please, help with is!!!! "You have exceeded your GPU quota... " I have paid the Pro service of 9 Euros per month, and everything remains the same, I barely do four tasks and I get the restriction message that I have to wait many minutes or more than an hour. So why have I paid the 9 Euros a month, if everything remains the same, with the same restrictions as if it were a free account? I have written, but they do not respond to me. Please help me.


Newswire: A Large-Scale Structured Database of a Century of Historical
  News (2406.09490)

Very cool dataset for journalists and historians just dropped: 2.7 million unique public domain U.S. news wire articles (1878-1977) 📰🕰

This is a goldmine for tracking historical events & newspaper coverage trends! An example? If we still wonder whether gender diversity in the media is important... "Only 4.6% of disambiguated entity mentions refer to women, and the most mentioned woman is Golda Meir."

Bonus:
- Locations in these articles are georeferenced
- Topics are tagged using customized neural topic classification
- Named entities are recognized,
- Individuals are disambiguated to Wikipedia using a novel entity disambiguation model

Anyone thinking of cool AI projects with this data? Maybe tracking the spread of news stories over time & space?

𝐆𝐨 𝐝𝐞𝐞𝐩𝐞𝐫
👉 Digg into the dataset: https://huggingface.co/datasets/dell-research-harvard/newswire
 👉 Read the paper: https://huggingface.co/papers/2406.09490


Running billion parameter models, sometimes we forget what it all is! 🤔💡

Matrix multiplication 🧮✨

While there are multiple plays on memory management and caching to speed it up! 🏎💾⚡

The naive way of Matrix multiplication becomes even more fascinating the bigger these models get! 🤯📈

QKV for the win! 🏆🔑📚

GitHub: https://github.com/wentasah/mmul-anim
Slides: https://cw.fel.cvut.cz/wiki/_media/courses/b4m36esw/esw09_2019.pdf 📑🎓


Introducing the first two projects on the HFforLegal community: the 'Laws' dataset and the associated search tool based on @nreimers and @tomaarsen's Sentence Transformers library 🤗

The objective of these two tools is to centralize in a single format a set of rules from different countries and legal systems in order to facilitate NLP in the field of comparative law, enabling more accurate and comprehensive legal analysis across different jurisdictions 🌍

Link to the dataset : https://huggingface.co/datasets/HFforLegal/laws
Link to the space: https://huggingface.co/spaces/HFforLegal/laws-retrieval

We need your contributions to enrich this new knowledge base, and you will find in the 'Laws' dataset all the information you need to format your data and submit them to the appropriate split.


New cookbook!

I show to to make agentic RAG using Transformers Agents.

Compared to vanilla RAG, agentic RAG can:
✅ Reformulate the query
✅ Critique the retrived content to re-retrieve if needed

➡ Score increase of 8.5%! 💪 (Llama-3-70B-judge)

Read it here 👉 https://huggingface.co/learn/cookbook/agent_rag


Animate a portrait with a driving video. Lots of potential fun here 😅 https://huggingface.co/spaces/KwaiVGI/LivePortrait

prithivMLmods Ramikan-BR GPT007 victor osanseviero Yersel


Kolors with VLM support

I've built a space for using Kolors image generation model with captioner models and prompt enhancers.

- Space with VLM and Prompt Enhancer
 https://huggingface.co/spaces/gokaygokay/KolorsPlusPlus

- Original Space for model
https://huggingface.co/spaces/gokaygokay/Kolors

- Captioner VLMs
- https://huggingface.co/gokaygokay/sd3-long-captioner-v2

- https://huggingface.co/microsoft/Florence-2-base

- Prompt Enhancers
- https://huggingface.co/gokaygokay/Lamini-Prompt-Enchance-Long

- https://huggingface.co/gokaygokay/Lamini-Prompt-Enchance

John6666 ucsahin kramp osanseviero Ramikan-BR yoeldcd tenet Joseph717171 AtAndDev Wok


Hey everyone!

Our team just dropped something cool! 🎉 We've published a new paper on arxiv diving into the foundation model leaderboards across different platforms. We've analyzed the content, operational workflows, and common issues of these leaderboards. From this, we came up with two new concepts: Leaderboard Operations (LBOps) and leaderboard smells.

We also put together an awesome list with nearly 300 of the latest leaderboards, development tools, and publishing organizations. You can check it out here: https://github.com/SAILResearch/awesome-foundation-model-leaderboards

If you find it useful or interesting, give us a follow or drop a comment. We'd love to hear your thoughts and get your support! ✨

Link to the paper: https://arxiv.org/abs/2407.04065

zhiminy osanseviero louisbrulenaudet Tonic


Excited to announce the release of the community version of our guardrails: WalledGuard-C!

Feel free to use it—compared to Meta’s guardrails, it offers superior performance, being 4x faster. Most importantly, it's free for nearly any use!

Link: https://huggingface.co/walledai/walledguard-c

#AISafety

Mehyaar osanseviero RishabhBhardwaj prithivMLmods


Anyone knows how to use SD3 for inpaint task ? :)


BrainGPT - Fun Weekend Project:)
Getting creative with a sci-fi 3D point cloud model of the brain - you prompt the model with questions about AI research frameworks that were deeply inspired by parts of the brain, you get a response with related papers 😂


Yet Another Whisper Finetune

https://huggingface.co/MohamedRashad/Arabic-Whisper-CodeSwitching-Edition is a finetune of OpenAI Whisper Large V2 on https://huggingface.co/datasets/MohamedRashad/arabic-english-code-switching dataset.
This new finetune is capable of recognizing english words in arabic speech and transcribing the foreign words as it is.

Try it out here:
https://huggingface.co/spaces/MohamedRashad/Arabic-Whisper-CodeSwitching-Edition

MohamedRashad osanseviero nyuuzyou manoumhd99


Introducing: The P-FAF Swarm Encoder! Now that I know you can do it with Swarm algorithms, the sky is the limit, beebee! The hotness AI tools over the past month have been LLM models with self referential feedback. AKA, getting them to do stuff on their own, like create data, transform data, etc. What if I told you that was wholly inefficient to do? What if I could replicate the same things and run it on a calculator? Meet P-FAF Swarm Encoder, your first real world proof that this is beyond viable. Plenty more proof to come!

Colab Notebook to try it yourself:
https://colab.research.google.com/drive/14CRumDep0SEubEiG3DCOCuGNGeZcsUkO?usp=sharing


🤗 Hi HF community!
🤖 A vital question that every developer may have asked themselves in the last three years is: how can we improve AI code generation?
😇 In my last Community blog post, I talk about  Codium AI's AlphaCodium and how they tried to enhance LLMs coding  skills with flow engineering: https://huggingface.co/blog/as-cle-bert/repetita-iuvant-how-to-improve-ai-code-generation
🚀 Enjoy!


A new paper titled "STALL+: Boosting LLM-based Repository-level Code Completion with Static Analysis" shows the benefits of integrating static analysis with LLMs. (https://arxiv.org/abs/2406.10018)

Authors evaluate 4 key questions:

- How does each static analysis integration strategy perform in LLM-based repository-level code completion?
> They found that integrating static analysis in the prompting phase (especially with file-level dependencies) can achieve the substantially larger improvements than other phases.

- How do different combinations of integration strategies affect LLM-based repository-level code completion?
> Languages that are easier to analyze like Java show more improvements compared to dynamic languages like Python.

- How do static analysis integration strategies perform when compared or combined with RAG in LLM-based repository-level code completion?
> Static analysis and RAG are complementary and boost the overall accuracy.

-  What are the online costs of different integration strategies in LLM-based repository-level code completion?
> Combining prompting-phase static analysis and RAG is the best option for cost-effectiveness.

In my @owasp App Sec keynote last year, I had described how one can do static analysis augmented generation (SaAG) to boost the accuracy of LLM based patches for vulnerability remediation. (you can see the talk here - https://www.youtube.com/watch?v=Cw4-ZnUNVLs)


Caffe 2 started it, TensorFlow brought it to the masses but PyTorch perfected it! ✨

Good folks at PyTorch are just wizards 🧙♂. Along with producing one of the best Deep Learning libraries of all time, they just dropped the "PyTorch Documentary" 🎥

A must-watch! It covers the beginning to now:
- Caffe 2 ☕
- Torch (with Lua) 🔥
- Tensorflow 🔄
- PyTorch 🔥
- Taffe IR (later became ONNX - Open Neural Network Exchange) 🔧

Full Official PyTorch Documentary: Powering the AI Revolution: https://youtu.be/rgP_LBtaUEc

Interesting quote: "PyTorch does not fight for the fastest performance but the easiest user experience!" 🌟

That's what Python 🐍 feels like...

You got to thank @Meta for open-sourcing it! 🤝


🧠 How to create more diverse, realistic synthetic AI training data?

@TencentAIGC-Lab AI Lab created @proj-persona, a vast collection of 1 billion diverse personas, to help create synthetic data with LLMs that encapsulate a wide array of perspectives, knowledge, experiences, interests, and professions.

These personas were created with automatically curated data, representing approximately 13% of the world’s total population.

💡 The authors argue that integrating a persona into data synthesis prompts effectively steers LLMs to adopt specific perspectives, creating unique and relevant synthetic data with minimal effort.

They showcased various practical applications of Persona Hub to demonstrate its effectiveness and versatility in various synthetic data creation scenarios: mathematical and logical reasoning problems, simulating diverse user requests and prompts for LLMs, generating informative and detailed text content across various topics, and more.

🚀 It's one of the trending datasets on Hugging Face. Digging into it is quite fun! I found one that reminds me of several people I know: "A journalist who covers technology and innovation in the print and digital media industries." It helped generate the prompt attached to this post (about which I'd be curious to know your answers 😉).

Synthetic data is a hot topic in AI. It will be interesting to see if this research could help make LLMs more robust, versatile, and capable of handling a wide array of real-world scenarios.

👉Explore the dataset: https://huggingface.co/datasets/proj-persona/PersonaHub
👉 Read the paper: https://arxiv.org/pdf/2406.20094

proj-persona ZeroWw mahami John6666 vikas louisbrulenaudet samsen5


I updated https://huggingface.co/spaces/nroggendorff/llava to be even worse. The point of this space is to point out how bad the base model for Llava is, and how without an image it struggles quite a bit. In the new update, from my testing, even if you ask the model about the image, it won't be able to tell you. I experimented with quite a few things, including an image where all the values are 0 (shoutout np.zeros), an image with the most generic portrait photo I could think of (black hair, brown eyes, plain white shirt, etc..) (generated with https://huggingface.co/spaces/nroggendorff/epicrealismxl), an image that has text that reads "The image you are looking for is unavailable", and much.. much more.. *sigh*. The image I'm ending up with seems to work best. I don't think it would work with other multi-modal LLMs, but I'm not sure. If you find a problem, feel free to message me on the [huggingface discord](http://hf.co/join/discord), or open a pull request.


📢 Delighted to share the most recent and valuable contributions to the book-related NLP domain 💎 To push forward deeper understanding of the characters 👨👩👧👦 from literature novel books itself 📖 by machine learning models 🤖, releasing the most-accessible version v1.0 of the related workflow, adopted for ParlAI 🦜 agents

🌟 https://github.com/nicolay-r/book-persona-retriever/tree/v1.0

Feel free to follow / share / comment in order to advance the related direction!


AI’s Cognitive Mirror: The Illusion of Consciousness in the Digital Age

https://empereur-pirate.medium.com/ais-cognitive-mirror-the-illusion-of-consciousness-in-the-digital-age-46f3ddae60a6

This text explores the philosophical implications of AI's potential for spiritual consciousness. It argues that AI, while capable of complex thought processes, lacks true self-awareness due to the absence of sensory perception. The article discusses how human consciousness develops through sensory experiences, contrasting this with AI's purely computational nature. It examines AI as a mirror of human thought, capable of impressive language processing and knowledge synthesis, yet  different from human psyche. The text also touches on the spiritual and ethical dimensions of AI development, drawing parallels with historical technological revolutions and religious symbolism. It concludes by considering AI's potential role in decision-making processes and its impact on various fields, while emphasizing the need for ethical safeguards and critical evaluation of AI-generated content.


Just released the GitVerse Code Dataset - https://huggingface.co/datasets/nyuuzyou/gitverse-code.

📊 Dataset highlights:
- 30 GB of unique code extracted from over 400 GB of analyzed data
- 9,014 repositories
- 2,804,216 unique code files
- 419 different file types
- Multilingual: various programming languages

🌐 Sourced from GitVerse, a Russian GitHub alternative opened in 2024.

Let me know your thoughts.

ZeroWw JoPmt nhannguyen1992 DeathGodlike nyuuzyou


Kolors: Effective Training of Diffusion Model for Photorealistic Text-to-Image Synthesis

Kolors is a large-scale text-to-image generation model based on latent diffusion, developed by the Kuaishou Kolors team.

Hugging Face Spaces
- https://huggingface.co/spaces/gokaygokay/Kolors

Model Page
- https://huggingface.co/Kwai-Kolors/Kolors


Really got amazing results with our InstantID next level. Tutorial will be published soon hopefully. This is the best ever 0-shot likeliness and high quality I have ever got.

prithivMLmods GPT007 rreed-pha NHLOCAL cnmoro Kkordik


Remember the recently released GLM-4 from Tsinghua University 🎉

Now we have an open-source version of it continuously trained for multilingual code generation! 🌍

It beats CodeLlama 70B (almost 7x size) and is competitive with DeepSeek Coder 33B and Qwen 2 💪

Just like almost every other coding model, it has a 128K context 📜

Supports:
- Code completion 🖋
- Code generation 🛠
- Code interpreter 💡
- Web search 🔍
- Function call 📞
- Repository-level code Q&A 🗂

Benchmarks 48.9 and 40.4 for the complete and instruct tasks of BigCodeBench 📊

It still falls behind DeepSeek-Coder-V2. While this might have fewer parameters, DSC V2 is a MoE model with only ~2B active parameters 🤔

Good to see more efficient coding LLMs, but DeepSeek-Coder-V2 is just too good 🏆

Codel: https://github.com/THUDM/CodeGeeX4
Model weights: https://huggingface.co/THUDM/codegeex4-all-9b


I believe in order to make models reach Human-Level Learning, serious students can start by developing an intelligent neuromorphic agent. We develop an intelligent agent and make it learn about grammar patterns as well as about different word categories through symbolic representations, following which we dwell into making the agent learn about other rules of the Language.

In parallel with grammar learning, the agent would also use language grounding techniques to link words to their sensory representations and abstract concepts which would mean the agent learns about the word meanings, synonyms, antonyms, and semantic relationships from both textual data as well as perceptual experiences.

The result would be the agent developing a rich lexicon and conceptual knowledge base that underlies its language understanding as well as generation. With this basic knowledge of grammar and word meanings, the agent can then learn to synthesize words and phrases so as to express specific ideas or concepts. Building on this, the agent would then learn how to generate complete sentences which the agent would continuously refine and improve. Eventually the agent would learn how to generate sequence of sentences in the form of dialogues or narratives, taking into account context, goals, as well as user-feedback.

I believe that by gradually learning how to improve their responses, the agent would gradually also acquire the ability to generate coherent, meaningful, and contextually appropriate language. This would allow them to reason without hallucinating which LLMs struggle at.

Developing such agents would not require a lot of compute and the code would be simple & easy to understand. It will definitely introduce everyone to symbolic AI and making agents which are good at reasoning tasks. Thus solving a crucial problem with LLMs. We have used a similar architecture to make our model learn constantly. Do sign up as we start opening access  next week at https://octave-x.com/


ColPali: A new approach to efficient and intelligent document retrieval 🚀

Our latest research paper, "ColPali: Efficient Document Retrieval with Vision Language Models," introduces a groundbreaking approach to large-scale visual document analysis. By leveraging Vision Language Models (VLMs), we have created a new framework for document retrieval that's both powerful and efficient.

Key Insights:
💡 ColPali combines ColBERT's multi-vector strategy with VLMs' document understanding capabilities
⚙ ColPali is based on PaliGemma-3B (SigLIP, Gemma-2B) + a linear projection layer and is trained to maximize the similarity between the document and the query embeddings
📊 The Vision Document Retrieval benchmark (ViDoRe) is a challenging dataset that spans various industry topics and aims at matching real-life retrieval scenarios
🏆 ColPali outperforms existing models on all datasets in ViDoRe (average NDCG@5 of 81.3% vs 67.0% for the best baseline model)
⚡ ColPali is faster at document embedding compared to traditional PDF parser pipelines, making ColPali viable for industrial use
🔍 ColPali is highly interpretable thanks to patch-based similarity maps

Dive deeper into ColPali and explore our resources:
📑 Full paper: arxiv.org/abs/2407.01449
🛠 Datasets, model weights, evaluation code, leaderboard, demos: huggingface.co/vidore

Shoutout to my amazing co-authors Manuel Faysse (@manu) and Hugues Sibille (@HugSib). We are grateful for the invaluable feedback from Bilel Omrani, Gautier Viaud, Celine Hudelot, and Pierre Colombo. This work is sponsored by ILLUIN Technology. ✨