text
Single Block / Layer FLUX LoRA Training Research Results and LoRA Network Alpha Change Impact With LoRA Network Rank Dimension
Single Block / Layer FLUX LoRA Training Research Results and LoRA Network Alpha Change Impact With LoRA Network Rank Dimension
new_line
new_line
text
Full article posted here : 
Full article posted here : 
link
https://medium.com/@furkangozukara/single-block-layer-flux-lora-training-research-results-and-lora-network-alpha-change-impact-with-e713cc89c567
https://medium.com/@furkangozukara/single-block-layer-flux-lora-training-research-results-and-lora-network-alpha-change-impact-with-e713cc89c567
new_line
new_line
text
Conclusions
Conclusions
new_line
text
As expected, as you train lesse parameters e.g. LoRA vs Full Fine Tuning or Single Blocks LoRA vs all Blocks LoRA, your quality get reduced
As expected, as you train lesse parameters e.g. LoRA vs Full Fine Tuning or Single Blocks LoRA vs all Blocks LoRA, your quality get reduced
new_line
text
Of course you earn some extra VRAM memory reduction and also some reduced size on the disk
Of course you earn some extra VRAM memory reduction and also some reduced size on the disk
new_line
text
Moreover, lesser parameters reduces the overfitting and realism of the FLUX model, so if you are into stylized outputs like comic, it may work better
Moreover, lesser parameters reduces the overfitting and realism of the FLUX model, so if you are into stylized outputs like comic, it may work better
new_line
text
Furthermore, when you reduce LoRA Network Rank, keep original Network Alpha unless you are going to do a new Learning Rate research
Furthermore, when you reduce LoRA Network Rank, keep original Network Alpha unless you are going to do a new Learning Rate research
new_line
text
Finally, very best and least overfitting is achieved with full Fine Tuning
Finally, very best and least overfitting is achieved with full Fine Tuning
new_line
text
Check figure 3 and figure 4 last columns — I make extracted LoRA Strength / Weight 1.1 instead of 1.0
Check figure 3 and figure 4 last columns — I make extracted LoRA Strength / Weight 1.1 instead of 1.0
new_line
text
Full fine tuning configs and instructions > 
Full fine tuning configs and instructions > 
link
https://www.patreon.com/posts/112099700
https://www.patreon.com/posts/112099700
new_line
text
Second best one is extracting a LoRA from Fine Tuned model if you need a LoRA
Second best one is extracting a LoRA from Fine Tuned model if you need a LoRA
new_line
text
Check figure 3 and figure 4 last columns — I make extracted LoRA Strength / Weight 1.1 instead of 1.0
Check figure 3 and figure 4 last columns — I make extracted LoRA Strength / Weight 1.1 instead of 1.0
new_line
text
Extract LoRA guide (public article) : 
Extract LoRA guide (public article) : 
link
https://www.patreon.com/posts/112335162
https://www.patreon.com/posts/112335162
new_line
text
Third is doing a all layers regular LoRA training
Third is doing a all layers regular LoRA training
new_line
text
Full guide, configs and instructions > 
Full guide, configs and instructions > 
link
https://www.patreon.com/posts/110879657
https://www.patreon.com/posts/110879657
new_line
text
And the worst quality is training lesser blocks / layers with LoRA
And the worst quality is training lesser blocks / layers with LoRA
new_line
text
Full configs are included in > 
Full configs are included in > 
link
https://www.patreon.com/posts/110879657
https://www.patreon.com/posts/110879657
new_line
text
So how much VRAM and Speed single block LoRA training brings?
So how much VRAM and Speed single block LoRA training brings?
new_line
text
All layers 16 bit is 27700 MB (4.85 second / it) and 1 single block is 25800 MB (3.7 second / it)
All layers 16 bit is 27700 MB (4.85 second / it) and 1 single block is 25800 MB (3.7 second / it)
new_line
text
All layers 8 bit is 17250 MB (4.85 second / it) and 1 single block is 15700 MB (3.8 second / it)
All layers 8 bit is 17250 MB (4.85 second / it) and 1 single block is 15700 MB (3.8 second / it)
new_line
text
Image Raw Links
Image Raw Links
new_line
text
Figure 0 : 
Figure 0 : 
resource
https://huggingface.co/MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests/resolve/main/Figure_0.jfif
model
MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
https://huggingface.co/MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests/resolve/main/Figure_0.jfif
new_line
Single Block / Layer FLUX LoRA Training Research Results and LoRA Network Alpha Change Impact With LoRA Network Rank Dimension

Full article posted here : https://medium.com/@furkangozukara/single-block-layer-flux-lora-training-research-results-and-lora-network-alpha-change-impact-with-e713cc89c567

Conclusions
As expected, as you train lesse parameters e.g. LoRA vs Full Fine Tuning or Single Blocks LoRA vs all Blocks LoRA, your quality get reduced
Of course you earn some extra VRAM memory reduction and also some reduced size on the disk
Moreover, lesser parameters reduces the overfitting and realism of the FLUX model, so if you are into stylized outputs like comic, it may work better
Furthermore, when you reduce LoRA Network Rank, keep original Network Alpha unless you are going to do a new Learning Rate research
Finally, very best and least overfitting is achieved with full Fine Tuning
Check figure 3 and figure 4 last columns — I make extracted LoRA Strength / Weight 1.1 instead of 1.0
Full fine tuning configs and instructions > https://www.patreon.com/posts/112099700
Second best one is extracting a LoRA from Fine Tuned model if you need a LoRA
Check figure 3 and figure 4 last columns — I make extracted LoRA Strength / Weight 1.1 instead of 1.0
Extract LoRA guide (public article) : https://www.patreon.com/posts/112335162
Third is doing a all layers regular LoRA training
Full guide, configs and instructions > https://www.patreon.com/posts/110879657
And the worst quality is training lesser blocks / layers with LoRA
Full configs are included in > https://www.patreon.com/posts/110879657
So how much VRAM and Speed single block LoRA training brings?
All layers 16 bit is 27700 MB (4.85 second / it) and 1 single block is 25800 MB (3.7 second / it)
All layers 8 bit is 17250 MB (4.85 second / it) and 1 single block is 15700 MB (3.8 second / it)
Image Raw Links
Figure 0 : https://huggingface.co/MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests/resolve/main/Figure_0.jfif

🚀
MonsterMMORPG John6666 jaisanrobert
👍
MonsterMMORPG multimodalart
🤯
MonsterMMORPG jaisanrobert
🔥
MonsterMMORPG
👀
MonsterMMORPG
❤️
MonsterMMORPG
🤗
MonsterMMORPG
😎
MonsterMMORPG
➕
MonsterMMORPG
🧠
MonsterMMORPG
🤝
MonsterMMORPG