sayakpaul HF staff commited on
Commit
ebefd78
·
verified ·
1 Parent(s): 84aad84

End of training

Browse files
README.md ADDED
@@ -0,0 +1,95 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: genmo/mochi-1-preview
3
+ library_name: diffusers
4
+ license: apache-2.0
5
+ instance_prompt: A black and white animated scene unfolds with an anthropomorphic
6
+ goat surrounded by musical notes and symbols, suggesting a playful environment.
7
+ Mickey Mouse appears, leaning forward in curiosity as the goat remains still. The
8
+ goat then engages with Mickey, who bends down to converse or react. The dynamics
9
+ shift as Mickey grabs the goat, potentially in surprise or playfulness, amidst a
10
+ minimalistic background. The scene captures the evolving relationship between the
11
+ two characters in a whimsical, animated setting, emphasizing their interactions
12
+ and emotions
13
+ widget:
14
+ - text: A black and white animated scene unfolds with an anthropomorphic goat surrounded
15
+ by musical notes and symbols, suggesting a playful environment. Mickey Mouse appears,
16
+ leaning forward in curiosity as the goat remains still. The goat then engages
17
+ with Mickey, who bends down to converse or react. The dynamics shift as Mickey
18
+ grabs the goat, potentially in surprise or playfulness, amidst a minimalistic
19
+ background. The scene captures the evolving relationship between the two characters
20
+ in a whimsical, animated setting, emphasizing their interactions and emotions
21
+ output:
22
+ url: final_video_0.mp4
23
+ tags:
24
+ - text-to-video
25
+ - diffusers-training
26
+ - diffusers
27
+ - lora
28
+ - mochi-1-preview
29
+ - mochi-1-preview-diffusers
30
+ - template:sd-lora
31
+ ---
32
+
33
+ <!-- This model card has been generated automatically according to the information the training script had access to. You
34
+ should probably proofread and complete it, then remove this comment. -->
35
+
36
+
37
+ # Mochi-1 Preview LoRA Finetune
38
+
39
+ <Gallery />
40
+
41
+ ## Model description
42
+
43
+ This is a lora finetune of the Mochi-1 preview model `genmo/mochi-1-preview`.
44
+
45
+ The model was trained using [CogVideoX Factory](https://github.com/a-r-r-o-w/cogvideox-factory) - a repository containing memory-optimized training scripts for the CogVideoX and Mochi family of models using [TorchAO](https://github.com/pytorch/ao) and [DeepSpeed](https://github.com/microsoft/DeepSpeed). The scripts were adopted from [CogVideoX Diffusers trainer](https://github.com/huggingface/diffusers/blob/main/examples/cogvideo/train_cogvideox_lora.py).
46
+
47
+ ## Download model
48
+
49
+ [Download LoRA](sayakpaul/mochi-lora/tree/main) in the Files & Versions tab.
50
+
51
+ ## Usage
52
+
53
+ Requires the [🧨 Diffusers library](https://github.com/huggingface/diffusers) installed.
54
+
55
+ ```py
56
+ from diffusers import MochiPipeline
57
+ from diffusers.utils import export_to_video
58
+ import torch
59
+
60
+ pipe = MochiPipeline.from_pretrained("genmo/mochi-1-preview")
61
+ pipe.load_lora_weights("CHANGE_ME")
62
+ pipe.enable_model_cpu_offload()
63
+
64
+ with torch.autocast("cuda", torch.bfloat16):
65
+ video = pipe(
66
+ prompt="CHANGE_ME",
67
+ guidance_scale=6.0,
68
+ num_inference_steps=64,
69
+ height=480,
70
+ width=848,
71
+ max_sequence_length=256,
72
+ output_type="np"
73
+ ).frames[0]
74
+ export_to_video(video)
75
+ ```
76
+
77
+ For more details, including weighting, merging and fusing LoRAs, check the [documentation](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters) on loading LoRAs in diffusers.
78
+
79
+
80
+
81
+ ## Intended uses & limitations
82
+
83
+ #### How to use
84
+
85
+ ```python
86
+ # TODO: add an example code snippet for running this diffusion pipeline
87
+ ```
88
+
89
+ #### Limitations and bias
90
+
91
+ [TODO: provide examples of latent issues and potential remediations]
92
+
93
+ ## Training details
94
+
95
+ [TODO: describe the data used to train the model]
final_video_0.mp4 ADDED
Binary file (84.2 kB). View file
 
pytorch_lora_weights.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:23ee7012c754ddabe91b25152bfc2357613e145c9a50f09de317206ef2861aff
3
+ size 76072152
test_video_0_A_black_and_white_animate.mp4 ADDED
Binary file (84.2 kB). View file
 
validation_video_0_A_black_and_white_animate.mp4 ADDED
Binary file (80.9 kB). View file