David Burnett commited on
Commit
d69a081
·
1 Parent(s): 855a122

Update README

Browse files
Files changed (1) hide show
  1. README.md +49 -0
README.md CHANGED
@@ -1,3 +1,52 @@
1
  ---
2
  license: openrail++
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: openrail++
3
  ---
4
+
5
+ This Repo contains a diffusers format version of the PixArt-Sigma Repos
6
+ PixArt-alpha/pixart_sigma_sdxlvae_T5_diffusers
7
+ PixArt-alpha/PixArt-Sigma-XL-2-1024-MS
8
+ with the models loaded and saved in fp16 and bf16 formats, roughly halfing their sizes.
9
+ It can be used where download bandwith, memory or diskspace are relatively low, a T4 Colab instance for example.
10
+
11
+ To use in a diffusers script you currently(15/04/2024) need to use a Source distribution of Diffusers
12
+ and an extra 'patch' from the PixArt0Alpha's teams Sigma Github repo
13
+
14
+ A simple Colab notebook can be found at https://github.com/Vargol/StableDiffusionColabs/blob/main/PixArt/PixArt_Sigma.ipynb
15
+
16
+ a Diffusers script looks like this.
17
+
18
+ ```py
19
+ import random
20
+ import sys
21
+ import torch
22
+ from diffusers import Transformer2DModel
23
+ from scripts.diffusers_patches import pixart_sigma_init_patched_inputs, PixArtSigmaPipeline
24
+
25
+ assert getattr(Transformer2DModel, '_init_patched_inputs', False), "Need to Upgrade diffusers: pip install git+https://github.com/huggingface/diffusers"
26
+ setattr(Transformer2DModel, '_init_patched_inputs', pixart_sigma_init_patched_inputs)
27
+ device = 'mps'
28
+ weight_dtype = torch.bfloat16
29
+
30
+ pipe = PixArtSigmaPipeline.from_pretrained(
31
+ "/Volumes/SSD2TB/AI/Repos/PixArt-Sigma_16bit",
32
+ torch_dtype=weight_dtype,
33
+ variant="fp16",
34
+ use_safetensors=True,
35
+ )
36
+
37
+ # Enable memory optimizations.
38
+ # pipe.enable_model_cpu_offload()
39
+ pipe.to(device)
40
+
41
+ prompt = "Cinematic science fiction film still.A cybernetic demon awaits her friend in a bar selling flaming oil drinks. The barman is a huge tree being, towering over the demon"
42
+
43
+ for i in range(4):
44
+
45
+ seed = random.randint(0, sys.maxsize)
46
+ generator = torch.Generator("mps").manual_seed(seed);
47
+
48
+
49
+ image = pipe(prompt, generator=generator, num_iferencenum_inference_steps=40).images[0]
50
+ image.save(f"pas_{seed}.png")a
51
+ ```
52
+