Update README.md
Browse files
README.md
CHANGED
@@ -1,60 +1,62 @@
|
|
1 |
-
---
|
2 |
-
license: apache-2.0
|
3 |
-
language:
|
4 |
-
- en
|
5 |
-
tags:
|
6 |
-
- cogvideox
|
7 |
-
- video-generation
|
8 |
-
- video-to-video
|
9 |
-
- controlnet
|
10 |
-
- diffusers
|
11 |
-
---
|
12 |
-
|
13 |
-
# CogvideoX-5b Controlnet Extention
|
14 |
-
|
15 |
-
|
16 |
-
|
17 |
-
|
18 |
-
###
|
19 |
-
|
20 |
-
|
21 |
-
|
22 |
-
|
23 |
-
|
24 |
-
|
25 |
-
|
26 |
-
|
27 |
-
|
28 |
-
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
```
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
|
40 |
-
|
41 |
-
|
42 |
-
--
|
43 |
-
--
|
44 |
-
--
|
45 |
-
|
46 |
-
|
47 |
-
|
48 |
-
|
49 |
-
|
50 |
-
|
51 |
-
|
52 |
-
--
|
53 |
-
|
54 |
-
|
55 |
-
|
56 |
-
|
57 |
-
|
58 |
-
|
59 |
-
|
|
|
|
|
60 |
<p>Issues should be raised directly in the repository. For professional support and recommendations please <a>[email protected]</a>.</p>
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
language:
|
4 |
+
- en
|
5 |
+
tags:
|
6 |
+
- cogvideox
|
7 |
+
- video-generation
|
8 |
+
- video-to-video
|
9 |
+
- controlnet
|
10 |
+
- diffusers
|
11 |
+
---
|
12 |
+
|
13 |
+
# CogvideoX-5b Controlnet Extention
|
14 |
+
|
15 |
+
|
16 |
+
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/63fde49f6315a264aba6a7ed/pdqauAcmf3d-v1mmu4UeR.mp4"></video>
|
17 |
+
|
18 |
+
### (Warning) This is raw version of controlnet. Better version will be published soon.
|
19 |
+
|
20 |
+
### How to
|
21 |
+
Clone repo
|
22 |
+
```bash
|
23 |
+
git clone https://github.com/TheDenk/cogvideox-controlnet.git
|
24 |
+
cd cogvideox-controlnet
|
25 |
+
```
|
26 |
+
|
27 |
+
Create venv
|
28 |
+
```bash
|
29 |
+
python -m venv venv
|
30 |
+
source venv/bin/activate
|
31 |
+
```
|
32 |
+
|
33 |
+
Install requirements
|
34 |
+
```bash
|
35 |
+
pip install -r requirements.txt
|
36 |
+
```
|
37 |
+
|
38 |
+
### Inference examples
|
39 |
+
#### Inference with cli
|
40 |
+
```bash
|
41 |
+
python -m inference.cli_demo \
|
42 |
+
--video_path "resources/car.mp4" \
|
43 |
+
--prompt "The camera follows behind red car. Car is surrounded by a panoramic view of the vast, azure ocean. Seagulls soar overhead, and in the distance, a lighthouse stands sentinel, its beam cutting through the twilight. The scene captures a perfect blend of adventure and serenity, with the car symbolizing freedom on the open sea." \
|
44 |
+
--controlnet_type "canny" \
|
45 |
+
--base_model_path THUDM/CogVideoX-5b \
|
46 |
+
--controlnet_model_path TheDenk/cogvideox-5b-controlnet-canny-v1
|
47 |
+
```
|
48 |
+
|
49 |
+
#### Inference with Gradio
|
50 |
+
```bash
|
51 |
+
python -m inference.gradio_web_demo \
|
52 |
+
--controlnet_type "canny" \
|
53 |
+
--base_model_path THUDM/CogVideoX-5b \
|
54 |
+
--controlnet_model_path TheDenk/cogvideox-5b-controlnet-canny-v1
|
55 |
+
```
|
56 |
+
|
57 |
+
|
58 |
+
## Acknowledgements
|
59 |
+
Original code and models [CogVideoX](https://github.com/THUDM/CogVideo/tree/main).
|
60 |
+
|
61 |
+
## Contacts
|
62 |
<p>Issues should be raised directly in the repository. For professional support and recommendations please <a>[email protected]</a>.</p>
|