text
The Stable Diffusion 3 research paper broken down, including some overlooked details! 📝
The Stable Diffusion 3 research paper broken down, including some overlooked details! 📝
new_line
new_line
text
Model
Model
new_line
text
📏 2 base model variants mentioned: 2B and 8B sizes
📏 2 base model variants mentioned: 2B and 8B sizes
new_line
new_line
text
📐 New architecture in all abstraction levels: 
📐 New architecture in all abstraction levels: 
new_line
text
- 🔽 UNet; ⬆️ Multimodal Diffusion Transformer, bye cross attention 👋
- 🔽 UNet; ⬆️ Multimodal Diffusion Transformer, bye cross attention 👋
new_line
text
- 🆕 Rectified flows for the diffusion process
- 🆕 Rectified flows for the diffusion process
new_line
text
- 🧩 Still a Latent Diffusion Model
- 🧩 Still a Latent Diffusion Model
new_line
new_line
text
📄 3 text-encoders: 2 CLIPs, one T5-XXL; plug-and-play: removing the larger one maintains competitiveness
📄 3 text-encoders: 2 CLIPs, one T5-XXL; plug-and-play: removing the larger one maintains competitiveness
new_line
new_line
text
🗃️ Dataset was deduplicated with SSCD which helped with memorization (no more details about the dataset tho) 
🗃️ Dataset was deduplicated with SSCD which helped with memorization (no more details about the dataset tho) 
new_line
new_line
text
Variants
Variants
new_line
text
🔁 A DPO fine-tuned model showed great improvement in prompt understanding and aesthetics
🔁 A DPO fine-tuned model showed great improvement in prompt understanding and aesthetics
new_line
text
✏️ An Instruct Edit 2B model was trained, and learned how to do text-replacement 
✏️ An Instruct Edit 2B model was trained, and learned how to do text-replacement 
new_line
new_line
text
Results
Results
new_line
text
✅ State of the art in automated evals for composition and prompt understanding 
✅ State of the art in automated evals for composition and prompt understanding 
new_line
text
✅ Best win rate in human preference evaluation for prompt understanding, aesthetics and typography (missing some details on how many participants and the design of the experiment)
✅ Best win rate in human preference evaluation for prompt understanding, aesthetics and typography (missing some details on how many participants and the design of the experiment)
new_line
new_line
text
Paper: 
Paper: 
link
https://stabilityai-public-packages.s3.us-west-2.amazonaws.com/Stable+Diffusion+3+Paper.pdf
https://stabilityai-public-packages.s3.us-west-2.amazonaws.com/Stable+Diffusion+3+Paper.pdf
text
The Stable Diffusion 3 research paper broken down, including some overlooked details! 📝

Model
📏 2 base model variants mentioned: 2B and 8B sizes

📐 New architecture in all abstraction levels: 
- 🔽 UNet; ⬆️ Multimodal Diffusion Transformer, bye cross attention 👋
- 🆕 Rectified flows for the diffusion process
- 🧩 Still a Latent Diffusion Model

📄 3 text-encoders: 2 CLIPs, one T5-XXL; plug-and-play: removing the larger one maintains competitiveness

🗃️ Dataset was deduplicated with SSCD which helped with memorization (no more details about the dataset tho) 

Variants
🔁 A DPO fine-tuned model showed great improvement in prompt understanding and aesthetics
✏️ An Instruct Edit 2B model was trained, and learned how to do text-replacement 

Results
✅ State of the art in automated evals for composition and prompt understanding 
✅ Best win rate in human preference evaluation for prompt understanding, aesthetics and typography (missing some details on how many participants and the design of the experiment)

Paper: https://stabilityai-public-packages.s3.us-west-2.amazonaws.com/Stable+Diffusion+3+Paper.pdf 
👍
osanseviero fffiloni victor mvaloatto clem BrigitteTousi WaveCut EtienneDosSantos samusenps Tonic chethan62 Eungbean MrForExample Obhee GPT007
❤️
clem BrigitteTousi samusenps leegao19 Tonic Bils gonduras
/avatars/ce597d8d2640c726473dd85ae8c5cdc7.svg
Lee Gao
leegao19
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1661165878439-noauth.jpeg
Valeriy Selitskiy
WaveCut
user
/avatars/38495afec0e0dc4c23308bb517bbc409.svg
NibbyNobNibber
MoveScores18
user