text
𝗠𝗲𝘁𝗮 𝘁𝗲𝗮𝗺 𝗷𝘂𝘀𝘁 𝗱𝗿𝗼𝗽𝗽𝗲𝗱 𝘁𝗵𝗲 𝗳𝗶𝗿𝘀𝘁 𝗪𝗮𝘁𝗲𝗿𝗺𝗮𝗿𝗸𝗶𝗻𝗴 𝗺𝗼𝗱𝗲𝗹 𝘁𝗵𝗮𝘁 𝗻𝗼𝘁 𝗲𝗱𝗶𝘁 𝗰𝗮𝗻 𝗯𝗿𝗲𝗮𝗸!🛡️
𝗠𝗲𝘁𝗮 𝘁𝗲𝗮𝗺 𝗷𝘂𝘀𝘁 𝗱𝗿𝗼𝗽𝗽𝗲𝗱 𝘁𝗵𝗲 𝗳𝗶𝗿𝘀𝘁 𝗪𝗮𝘁𝗲𝗿𝗺𝗮𝗿𝗸𝗶𝗻𝗴 𝗺𝗼𝗱𝗲𝗹 𝘁𝗵𝗮𝘁 𝗻𝗼𝘁 𝗲𝗱𝗶𝘁 𝗰𝗮𝗻 𝗯𝗿𝗲𝗮𝗸!🛡️
new_line
new_line
text
🤔 Ever heard of watermarking? It's a technique that allows you to mark in an image its original source. It's our best shield against AI-generated deepfakes, or content stolen from artists! 🎨 
🤔 Ever heard of watermarking? It's a technique that allows you to mark in an image its original source. It's our best shield against AI-generated deepfakes, or content stolen from artists! 🎨 
new_line
new_line
text
🎭 Watermarking systems are actually a pair of models: a watermark embedder that applies the watermark on the image, and its corresponding decoder that should detect the original watermark.
🎭 Watermarking systems are actually a pair of models: a watermark embedder that applies the watermark on the image, and its corresponding decoder that should detect the original watermark.
new_line
new_line
text
⛔ But current methods were very limited: they can only apply and detect the watermark on your image as a whole. So, if you're an attacker it's easy to break: just crop it! add text on top! or whatever, really, anything would work to break the watermark.
⛔ But current methods were very limited: they can only apply and detect the watermark on your image as a whole. So, if you're an attacker it's easy to break: just crop it! add text on top! or whatever, really, anything would work to break the watermark.
new_line
new_line
text
A team of researchers at Meta was not happy with this. 😤
A team of researchers at Meta was not happy with this. 😤
new_line
new_line
text
So to withstand real-world attacks, they decided to make a watermarking model that would also work on any sub-part of the image. It's a real paradigm shift: they consider watermarking not as an image classification task, but as an image segmentation task!
So to withstand real-world attacks, they decided to make a watermarking model that would also work on any sub-part of the image. It's a real paradigm shift: they consider watermarking not as an image classification task, but as an image segmentation task!
new_line
new_line
text
🏗️ 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲
🏗️ 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲
new_line
text
▸ The "Embedder" (a variational autoencoder + embedder, 1.1M parameters in total) encodes a n-bit message into a watermark signal that is added to the original image
▸ The "Embedder" (a variational autoencoder + embedder, 1.1M parameters in total) encodes a n-bit message into a watermark signal that is added to the original image
new_line
text
▸ [Only during training] The "Augmenter" randomly distorts the image: masks parts, crops, resizes, compresses. It's basically torture at this point.
▸ [Only during training] The "Augmenter" randomly distorts the image: masks parts, crops, resizes, compresses. It's basically torture at this point.
new_line
text
▸ The "Extractor" (a vision transformer, or ViT, with 96M parameters) then re-extracts the message from the distorted image, by predicting a (1+n) vector per pixel to predict the watermarked parts and decode corresponding messages.
▸ The "Extractor" (a vision transformer, or ViT, with 96M parameters) then re-extracts the message from the distorted image, by predicting a (1+n) vector per pixel to predict the watermarked parts and decode corresponding messages.
new_line
new_line
text
The performance blows existing models out of the water, they even created new tasks (segmentation-related) just to grok them!
The performance blows existing models out of the water, they even created new tasks (segmentation-related) just to grok them!
new_line
new_line
text
Gerat work 
Gerat work 
mention
@pierrefdz
pierrefdz
text
 and 
 and 
mention
@tomsander1998
tomsander1998
text
!
!
new_line
new_line
text
Paper here 👉 
Paper here 👉 
resource
https://huggingface.co/papers/2411.07231
paper
2411.07231
https://huggingface.co/papers/2411.07231
Watermark Anything with Localized Messages (2411.07231)
𝗠𝗲𝘁𝗮 𝘁𝗲𝗮𝗺 𝗷𝘂𝘀𝘁 𝗱𝗿𝗼𝗽𝗽𝗲𝗱 𝘁𝗵𝗲 𝗳𝗶𝗿𝘀𝘁 𝗪𝗮𝘁𝗲𝗿𝗺𝗮𝗿𝗸𝗶𝗻𝗴 𝗺𝗼𝗱𝗲𝗹 𝘁𝗵𝗮𝘁 𝗻𝗼𝘁 𝗲𝗱𝗶𝘁 𝗰𝗮𝗻 𝗯𝗿𝗲𝗮𝗸!🛡️

🤔 Ever heard of watermarking? It's a technique that allows you to mark in an image its original source. It's our best shield against AI-generated deepfakes, or content stolen from artists! 🎨 

🎭 Watermarking systems are actually a pair of models: a watermark embedder that applies the watermark on the image, and its corresponding decoder that should detect the original watermark.

⛔ But current methods were very limited: they can only apply and detect the watermark on your image as a whole. So, if you're an attacker it's easy to break: just crop it! add text on top! or whatever, really, anything would work to break the watermark.

A team of researchers at Meta was not happy with this. 😤

So to withstand real-world attacks, they decided to make a watermarking model that would also work on any sub-part of the image. It's a real paradigm shift: they consider watermarking not as an image classification task, but as an image segmentation task!

🏗️ 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲
▸ The "Embedder" (a variational autoencoder + embedder, 1.1M parameters in total) encodes a n-bit message into a watermark signal that is added to the original image
▸ [Only during training] The "Augmenter" randomly distorts the image: masks parts, crops, resizes, compresses. It's basically torture at this point.
▸ The "Extractor" (a vision transformer, or ViT, with 96M parameters) then re-extracts the message from the distorted image, by predicting a (1+n) vector per pixel to predict the watermarked parts and decode corresponding messages.

The performance blows existing models out of the water, they even created new tasks (segmentation-related) just to grok them!

Gerat work @pierrefdz and @tomsander1998!

Paper here 👉 https://huggingface.co/papers/2411.07231
👀
John6666 Aurelien-Morgan