Hugging Face's logo

Aratako
/
Mixtral-8x7B-Instruct-v0.1-upscaled

Text Generation
Transformers
Safetensors
mixtral
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
Aratako commited on
Commit
8da10a2
·
verified ·
1 Parent(s): 3e3b7c3

Update mergekit_config.yml

Browse files
Files changed (1) hide show
  1. mergekit_config.yml +25 -25
mergekit_config.yml CHANGED
@@ -1,25 +1,25 @@
1
- merge_method: passthrough
2
- slices:
3
- - sources:
4
- - model: ./Mixtral-8x7B-Instruct-v0.1
5
- layer_range: [0, 8]
6
- - sources:
7
- - model: ./Mixtral-8x7B-Instruct-v0.1
8
- layer_range: [4, 12]
9
- - sources:
10
- - model: ./Mixtral-8x7B-Instruct-v0.1
11
- layer_range: [8, 16]
12
- - sources:
13
- - model: ./Mixtral-8x7B-Instruct-v0.1
14
- layer_range: [12, 20]
15
- - sources:
16
- - model: ./Mixtral-8x7B-Instruct-v0.1
17
- layer_range: [16, 24]
18
- - sources:
19
- - model: ./Mixtral-8x7B-Instruct-v0.1
20
- layer_range: [20, 28]
21
- - sources:
22
- - model: ./Mixtral-8x7B-Instruct-v0.1
23
- layer_range: [24, 32]
24
- dtype: bfloat16
25
- tokenizer_source: base
 
1
+ merge_method: passthrough
2
+ slices:
3
+ - sources:
4
+ - model: mistralai/Mixtral-8x7B-Instruct-v0.1
5
+ layer_range: [0, 8]
6
+ - sources:
7
+ - model: mistralai/Mixtral-8x7B-Instruct-v0.1
8
+ layer_range: [4, 12]
9
+ - sources:
10
+ - model: mistralai/Mixtral-8x7B-Instruct-v0.1
11
+ layer_range: [8, 16]
12
+ - sources:
13
+ - model: mistralai/Mixtral-8x7B-Instruct-v0.1
14
+ layer_range: [12, 20]
15
+ - sources:
16
+ - model: mistralai/Mixtral-8x7B-Instruct-v0.1
17
+ layer_range: [16, 24]
18
+ - sources:
19
+ - model: mistralai/Mixtral-8x7B-Instruct-v0.1
20
+ layer_range: [20, 28]
21
+ - sources:
22
+ - model: mistralai/Mixtral-8x7B-Instruct-v0.1
23
+ layer_range: [24, 32]
24
+ dtype: bfloat16
25
+ tokenizer_source: base