wujunqiang
commited on
Commit
•
deb71be
1
Parent(s):
1fd8579
Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,6 @@ library_name: diffusers
|
|
10 |
---
|
11 |
|
12 |
|
13 |
-
|
14 |
Kolors-ControlNet-Canny weights and inference code
|
15 |
|
16 |
|
@@ -24,21 +23,12 @@ We provide two ControlNet weights and inference code based on Kolors-Basemodel:
|
|
24 |
<img src="demo1.png">
|
25 |
|
26 |
|
27 |
-
|
28 |
-
|
29 |
**2、ControlNet and IP-Adapter-Plus Demos**
|
30 |
|
31 |
We also support joint inference code between Kolors-IPadapter and Kolors-ControlNet.
|
32 |
|
33 |
-
|
34 |
<img src="demo2.png">
|
35 |
|
36 |
-
<br>
|
37 |
-
|
38 |
-
|
39 |
-
<br>
|
40 |
-
|
41 |
-
|
42 |
## <a name="Evaluation"></a>📊 Evaluation
|
43 |
To evaluate the performance of models, we compiled a test set of more than 200 images and text prompts. We invite several image experts to provide fair ratings for the generated results of different models. The experts rate the generated images based on four criteria: visual appeal, text faithfulness, conditional controllability, and overall satisfaction. Conditional controllability measures controlnet's ability to preserve spatial structure, while the other criteria follow the evaluation standards of BaseModel. The specific results are summarized in the table below, where Kolors-ControlNet achieved better performance in various criterias.
|
44 |
|
@@ -58,15 +48,12 @@ To evaluate the performance of models, we compiled a test set of more than 200 i
|
|
58 |
| SDXL-ControlNet-Canny | 3.35 | 3.77 | 4.26 | 4.5 |
|
59 |
| **Kolors-ControlNet-Depth** | **4.12** | **4.12** | **4.62** | **4.6** |
|
60 |
|
61 |
-
|
62 |
-
<font color=gray style="font-size:12px">*The [SDXL-ControlNet-Canny](https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0) and [SDXL-ControlNet-Depth](https://huggingface.co/diffusers/controlnet-depth-sdxl-1.0) load [DreamShaper-XL](https://civitai.com/models/112902?modelVersionId=351306) as backbone model.*</font>
|
63 |
-
|
64 |
<img src="compare_demo.png">
|
65 |
|
|
|
66 |
|
67 |
------
|
68 |
|
69 |
-
|
70 |
## <a name="Usage"></a>🛠️ Usage
|
71 |
|
72 |
### Requirements
|
@@ -115,7 +102,6 @@ python ./controlnet/sample_controlNet.py ./controlnet/assets/bird.png 一只颜
|
|
115 |
```
|
116 |
|
117 |
|
118 |
-
|
119 |
**c. Using depth ControlNet + IP-Adapter-Plus:**
|
120 |
|
121 |
If you intend to utilize the kolors-ip-adapter-plus, please ensure to download its corresponding model weights.
|
@@ -126,13 +112,8 @@ python ./controlnet/sample_controlNet_ipadapter.py ./controlnet/assets/woman_2.p
|
|
126 |
# The image will be saved to "controlnet/outputs/"
|
127 |
```
|
128 |
|
129 |
-
<br>
|
130 |
-
|
131 |
-
|
132 |
### Acknowledgments
|
133 |
- Thanks to [ControlNet](https://github.com/lllyasviel/ControlNet) for providing the codebase.
|
134 |
|
135 |
<br>
|
136 |
|
137 |
-
|
138 |
-
|
|
|
10 |
---
|
11 |
|
12 |
|
|
|
13 |
Kolors-ControlNet-Canny weights and inference code
|
14 |
|
15 |
|
|
|
23 |
<img src="demo1.png">
|
24 |
|
25 |
|
|
|
|
|
26 |
**2、ControlNet and IP-Adapter-Plus Demos**
|
27 |
|
28 |
We also support joint inference code between Kolors-IPadapter and Kolors-ControlNet.
|
29 |
|
|
|
30 |
<img src="demo2.png">
|
31 |
|
|
|
|
|
|
|
|
|
|
|
|
|
32 |
## <a name="Evaluation"></a>📊 Evaluation
|
33 |
To evaluate the performance of models, we compiled a test set of more than 200 images and text prompts. We invite several image experts to provide fair ratings for the generated results of different models. The experts rate the generated images based on four criteria: visual appeal, text faithfulness, conditional controllability, and overall satisfaction. Conditional controllability measures controlnet's ability to preserve spatial structure, while the other criteria follow the evaluation standards of BaseModel. The specific results are summarized in the table below, where Kolors-ControlNet achieved better performance in various criterias.
|
34 |
|
|
|
48 |
| SDXL-ControlNet-Canny | 3.35 | 3.77 | 4.26 | 4.5 |
|
49 |
| **Kolors-ControlNet-Depth** | **4.12** | **4.12** | **4.62** | **4.6** |
|
50 |
|
|
|
|
|
|
|
51 |
<img src="compare_demo.png">
|
52 |
|
53 |
+
<font color=gray style="font-size:12px">*The [SDXL-ControlNet-Canny](https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0) and [SDXL-ControlNet-Depth](https://huggingface.co/diffusers/controlnet-depth-sdxl-1.0) load [DreamShaper-XL](https://civitai.com/models/112902?modelVersionId=351306) as backbone model.*</font>
|
54 |
|
55 |
------
|
56 |
|
|
|
57 |
## <a name="Usage"></a>🛠️ Usage
|
58 |
|
59 |
### Requirements
|
|
|
102 |
```
|
103 |
|
104 |
|
|
|
105 |
**c. Using depth ControlNet + IP-Adapter-Plus:**
|
106 |
|
107 |
If you intend to utilize the kolors-ip-adapter-plus, please ensure to download its corresponding model weights.
|
|
|
112 |
# The image will be saved to "controlnet/outputs/"
|
113 |
```
|
114 |
|
|
|
|
|
|
|
115 |
### Acknowledgments
|
116 |
- Thanks to [ControlNet](https://github.com/lllyasviel/ControlNet) for providing the codebase.
|
117 |
|
118 |
<br>
|
119 |
|
|
|
|