Hugging Face's logo

mashleburneded
/
Mobiusv0.3

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
text-generation-inference
Inference Endpoints
Model card Files Community
mashleburneded commited on
Commit
90232c1
·
verified ·
1 Parent(s): 872484f

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +1 -0
  2. README.md +56 -0
  3. config.json +27 -0
  4. mergekit_config.yml +18 -0
  5. model-00001-of-00073.safetensors +3 -0
  6. model-00002-of-00073.safetensors +3 -0
  7. model-00003-of-00073.safetensors +3 -0
  8. model-00004-of-00073.safetensors +3 -0
  9. model-00005-of-00073.safetensors +3 -0
  10. model-00006-of-00073.safetensors +3 -0
  11. model-00007-of-00073.safetensors +3 -0
  12. model-00008-of-00073.safetensors +3 -0
  13. model-00009-of-00073.safetensors +3 -0
  14. model-00010-of-00073.safetensors +3 -0
  15. model-00011-of-00073.safetensors +3 -0
  16. model-00012-of-00073.safetensors +3 -0
  17. model-00013-of-00073.safetensors +3 -0
  18. model-00014-of-00073.safetensors +3 -0
  19. model-00015-of-00073.safetensors +3 -0
  20. model-00016-of-00073.safetensors +3 -0
  21. model-00017-of-00073.safetensors +3 -0
  22. model-00018-of-00073.safetensors +3 -0
  23. model-00019-of-00073.safetensors +3 -0
  24. model-00020-of-00073.safetensors +3 -0
  25. model-00021-of-00073.safetensors +3 -0
  26. model-00022-of-00073.safetensors +3 -0
  27. model-00023-of-00073.safetensors +3 -0
  28. model-00024-of-00073.safetensors +3 -0
  29. model-00025-of-00073.safetensors +3 -0
  30. model-00026-of-00073.safetensors +3 -0
  31. model-00027-of-00073.safetensors +3 -0
  32. model-00028-of-00073.safetensors +3 -0
  33. model-00029-of-00073.safetensors +3 -0
  34. model-00030-of-00073.safetensors +3 -0
  35. model-00031-of-00073.safetensors +3 -0
  36. model-00032-of-00073.safetensors +3 -0
  37. model-00033-of-00073.safetensors +3 -0
  38. model-00034-of-00073.safetensors +3 -0
  39. model-00035-of-00073.safetensors +3 -0
  40. model-00036-of-00073.safetensors +3 -0
  41. model-00037-of-00073.safetensors +3 -0
  42. model-00038-of-00073.safetensors +3 -0
  43. model-00039-of-00073.safetensors +3 -0
  44. model-00040-of-00073.safetensors +3 -0
  45. model-00041-of-00073.safetensors +3 -0
  46. model-00042-of-00073.safetensors +3 -0
  47. model-00043-of-00073.safetensors +3 -0
  48. model-00044-of-00073.safetensors +3 -0
  49. model-00045-of-00073.safetensors +3 -0
  50. model-00046-of-00073.safetensors +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - mistralai/Mistral-Nemo-Base-2407
4
+ - mistralai/Mistral-Large-Instruct-2407
5
+ - mistralai/Codestral-22B-v0.1
6
+ - mistralai/Mathstral-7B-v0.1
7
+ - nvidia/Mistral-NeMo-Minitron-8B-Instruct
8
+ library_name: transformers
9
+ tags:
10
+ - mergekit
11
+ - merge
12
+
13
+ ---
14
+ # merge
15
+
16
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
17
+
18
+ ## Merge Details
19
+ ### Merge Method
20
+
21
+ This model was merged using the passthrough merge method.
22
+
23
+ ### Models Merged
24
+
25
+ The following models were included in the merge:
26
+ * [mistralai/Mistral-Nemo-Base-2407](https://huggingface.co/mistralai/Mistral-Nemo-Base-2407)
27
+ * [mistralai/Mistral-Large-Instruct-2407](https://huggingface.co/mistralai/Mistral-Large-Instruct-2407)
28
+ * [mistralai/Codestral-22B-v0.1](https://huggingface.co/mistralai/Codestral-22B-v0.1)
29
+ * [mistralai/Mathstral-7B-v0.1](https://huggingface.co/mistralai/Mathstral-7B-v0.1)
30
+ * [nvidia/Mistral-NeMo-Minitron-8B-Instruct](https://huggingface.co/nvidia/Mistral-NeMo-Minitron-8B-Instruct)
31
+
32
+ ### Configuration
33
+
34
+ The following YAML configuration was used to produce this model:
35
+
36
+ ```yaml
37
+ dtype: float16
38
+ merge_method: passthrough
39
+ slices:
40
+ - sources:
41
+ - layer_range: [16, 32]
42
+ model: mistralai/Mistral-Large-Instruct-2407
43
+ - sources:
44
+ - layer_range: [20, 32]
45
+ model: nvidia/Mistral-NeMo-Minitron-8B-Instruct
46
+ - sources:
47
+ - layer_range: [24, 32]
48
+ model: mistralai/Mistral-Nemo-Base-2407
49
+ - sources:
50
+ - layer_range: [28, 32]
51
+ model: mistralai/Codestral-22B-v0.1
52
+ - sources:
53
+ - layer_range: [32, 32]
54
+ model: mistralai/Mathstral-7B-v0.1
55
+
56
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "mistralai/Mistral-Nemo-Base-2407",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 5120,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 14336,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 32,
17
+ "num_hidden_layers": 40,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "float16",
24
+ "transformers_version": "4.46.2",
25
+ "use_cache": true,
26
+ "vocab_size": 131072
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ dtype: float16
2
+ merge_method: passthrough
3
+ slices:
4
+ - sources:
5
+ - layer_range: [16, 32]
6
+ model: mistralai/Mistral-Large-Instruct-2407
7
+ - sources:
8
+ - layer_range: [20, 32]
9
+ model: nvidia/Mistral-NeMo-Minitron-8B-Instruct
10
+ - sources:
11
+ - layer_range: [24, 32]
12
+ model: mistralai/Mistral-Nemo-Base-2407
13
+ - sources:
14
+ - layer_range: [28, 32]
15
+ model: mistralai/Codestral-22B-v0.1
16
+ - sources:
17
+ - layer_range: [32, 32]
18
+ model: mistralai/Mathstral-7B-v0.1
model-00001-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c8d2bbe13a2b0867d821474c0fc762b2d20f19da6ce78128ba231c5e539f9f6f
3
+ size 268435584
model-00002-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c78ba980a2039f7d31d845e9f564715ba041cf4d90f57e5ed869c72cee3d754
3
+ size 805331192
model-00003-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d5c1eb8b6e3026e243f9e934079542b7f048b598f2e9968a2a914dd9c4d44db6
3
+ size 704643224
model-00004-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:523791982afe159431f478887571a89d08a7d4b7c33698744037c08ebdf8cec7
3
+ size 704643224
model-00005-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7581f4a8e849e9c605814702cbb6cddfdc39165f5b9ec29c8a66faa54136f46d
3
+ size 729833848
model-00006-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:242732b10569f8c004d9db159d064c3de8d0123c703499ec249a6766187b05bc
3
+ size 629170664
model-00007-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49467fb4dfdc3ef912d27ca0ab0626d113ae17a310cda5eab324772c570ff4a1
3
+ size 704643224
model-00008-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e08dc2329d9dcaef8ca5a4c6f01483e5f81ac214a7a8b0ac9310790c18498bb
3
+ size 704643224
model-00009-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99811d7bcd94c8224f59aa23b17c1847f7a9977f9d56998a0c877e21c13744cb
3
+ size 729833848
model-00010-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e37c7e712b0f56d68ff61b27f8a0d2cc2a5b12615be5de332fd44b69af79985
3
+ size 629170664
model-00011-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eaeb1a449b9a30a632ffb3d2b4cd37dac9e93c1e05ee6e115a0374354fd6b2f9
3
+ size 704643224
model-00012-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:625256834c9469f6d2a3b986ff559460be75b12e9cf7bd74dcc862006e8b1d5e
3
+ size 704643224
model-00013-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bcbf826baf23e14f0b7d2494445cf75547343a9a15df519d6692646909148a9c
3
+ size 729833848
model-00014-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad9a4a46376373bd596e582070f6ee0d77553a549d6d9abd96dde75da6bc0af2
3
+ size 629170664
model-00015-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78c0416de45aac7f2cfe9f7471d4354edf5d407daa7afd947b0ee1bed9a90ccf
3
+ size 704643224
model-00016-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8579ed0061b88cec7b6a93594ca22afd79d0c1b7535fb889bc5830f119e2f069
3
+ size 704643224
model-00017-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9081da6cb36ea8d95561c96a7de0024c90fb1e4971e161ae3d656773c4730c21
3
+ size 729833848
model-00018-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0cde3b2f7eda2c6f8ca257008f893fab7b4e2c693665a687416927f606883535
3
+ size 723550912
model-00019-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f71737e8e57237ad29981bbb584f51b78bb4207f0923868b6307236b316750a
3
+ size 799015168
model-00020-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:79d3cfd4ec1382aea9cde5bbe4096394fffd8b1bc67021dcd42f59225d0692f6
3
+ size 799015168
model-00021-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc471d66c477f9585ce07ed3032ada3805daf38ab3abe7b6501df0fc57253dc1
3
+ size 771785416
model-00022-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:89ddd12d52217db8899d037994cbeb1e744a61177bf0e3fb3eb1e238be3984a8
3
+ size 765494184
model-00023-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f2d11fae3a8d350020d238e9ee50e95870363f7300f60aecc911ecb62997351
3
+ size 799015168
model-00024-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6564b27c4182b840cd731b694e69d7101087465c0b85c7cd96328b78bac7f3e8
3
+ size 799015168
model-00025-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:388c96f0c1b8d23d58ea839b5fcf104a52801610321a5457c565ae8411ff837f
3
+ size 771785416
model-00026-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5e594dc39eeac3cb3afd42afd867f357f149d4792be1d703bf7d6d6a96520bbe
3
+ size 765494184
model-00027-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74a877f2b3100f0c7634164a1669475ad5e29b62802aba21e71cbdf71ce52f5d
3
+ size 799015168
model-00028-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:899860d68d9d1c07c2a32f7b93b27f3fab6a2e0600112f68003312a715ce3c2e
3
+ size 799015168
model-00029-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e8e4e7b7a8188c8dd4c932219145cd0ea3216bc64a0668170cab3499543abf99
3
+ size 771785416
model-00030-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:844dd9c362c98eef8eae748c56aa21daacb1e294ac02008c0a1472ae38354824
3
+ size 765494184
model-00031-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:137b3a9cd07bc45b1ca4fd82f8980e5573ca80cd9b57829b451a4f3985625527
3
+ size 799015168
model-00032-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2877ba81390aa16aba28785109e22066f4071834b5594f7c059063ad47fcb476
3
+ size 799015168
model-00033-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2df175e5a856fa1aca380d71adb33434c538ed8ef027badbc6921f198786322
3
+ size 771785416
model-00034-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d88041e6f72370f907732220da0b5eae85ea300599311a2b98ceba5e5507a77
3
+ size 912305288
model-00035-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ddcd371e8da93728073e547866234684b5bc046aac91610faf18e7b459e15d7f
3
+ size 945815928
model-00036-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f8ba479cc00ab97842c89fafd701c7da46ec1a644bb8e1680dd799716d9e75b
3
+ size 945815920
model-00037-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a1920f688a4762bb3cd0b647662874692933efeba13086dc6c85bc0204e12f5
3
+ size 824224800
model-00038-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6848dead6ee542ba440cdb88dfdc63e8a2ac808e23c2e5d031c6510db185d701
3
+ size 964734320
model-00039-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c11b2af7407870fa178ab2a4ebe516f7d167d364189384d0de2f31f4d5dd0ba8
3
+ size 945815928
model-00040-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6b85e13be5f8929cf37a8709d0caba0fb3dedd03ad72f1433efef3236a8079f
3
+ size 945815920
model-00041-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0d03930372ad94a72e32697a6cc661115e522229d7b90f41b40d7ca7811367e
3
+ size 824224800
model-00042-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee41d31eda84ebdd4bfd10593477df31634ea1e74c32611c9e2cfb3db0328087
3
+ size 964734312
model-00043-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f24d3e431cd2964eea814198ba2bfc7bfcd6a979df22c7de1128c16514b9409
3
+ size 945815928
model-00044-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef55473a431e8ec5925c203d5f1058b13ff676586dcee13fef4037a4d62ee54a
3
+ size 945815920
model-00045-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e6065ef91e4834365b2f64fdc97a4498f55f8f25cb38418cedc7b51cd3a8f3a3
3
+ size 824224824
model-00046-of-00073.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:55bc045b53cf1fbd41e7bde279cb23513895ab7824a4edc6201e638944e8a365
3
+ size 964734328