Hugging Face's logo

softwareweaver
/
Twilight-XL-195B

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
softwareweaver commited on
Commit
9de498b
·
verified ·
1 Parent(s): 226e7cd

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +37 -0
  2. config.json +27 -0
  3. mergekit_config.yml +9 -0
  4. model-00001-of-00080.safetensors +3 -0
  5. model-00002-of-00080.safetensors +3 -0
  6. model-00003-of-00080.safetensors +3 -0
  7. model-00004-of-00080.safetensors +3 -0
  8. model-00005-of-00080.safetensors +3 -0
  9. model-00006-of-00080.safetensors +3 -0
  10. model-00007-of-00080.safetensors +3 -0
  11. model-00008-of-00080.safetensors +3 -0
  12. model-00009-of-00080.safetensors +3 -0
  13. model-00010-of-00080.safetensors +3 -0
  14. model-00011-of-00080.safetensors +3 -0
  15. model-00012-of-00080.safetensors +3 -0
  16. model-00013-of-00080.safetensors +3 -0
  17. model-00014-of-00080.safetensors +3 -0
  18. model-00015-of-00080.safetensors +3 -0
  19. model-00016-of-00080.safetensors +3 -0
  20. model-00017-of-00080.safetensors +3 -0
  21. model-00018-of-00080.safetensors +3 -0
  22. model-00019-of-00080.safetensors +3 -0
  23. model-00020-of-00080.safetensors +3 -0
  24. model-00021-of-00080.safetensors +3 -0
  25. model-00022-of-00080.safetensors +3 -0
  26. model-00023-of-00080.safetensors +3 -0
  27. model-00024-of-00080.safetensors +3 -0
  28. model-00025-of-00080.safetensors +3 -0
  29. model-00026-of-00080.safetensors +3 -0
  30. model-00027-of-00080.safetensors +3 -0
  31. model-00028-of-00080.safetensors +3 -0
  32. model-00029-of-00080.safetensors +3 -0
  33. model-00030-of-00080.safetensors +3 -0
  34. model-00031-of-00080.safetensors +3 -0
  35. model-00032-of-00080.safetensors +3 -0
  36. model-00033-of-00080.safetensors +3 -0
  37. model-00034-of-00080.safetensors +3 -0
  38. model-00035-of-00080.safetensors +3 -0
  39. model-00036-of-00080.safetensors +3 -0
  40. model-00037-of-00080.safetensors +3 -0
  41. model-00038-of-00080.safetensors +3 -0
  42. model-00039-of-00080.safetensors +3 -0
  43. model-00040-of-00080.safetensors +3 -0
  44. model-00041-of-00080.safetensors +3 -0
  45. model-00042-of-00080.safetensors +3 -0
  46. model-00043-of-00080.safetensors +3 -0
  47. model-00044-of-00080.safetensors +3 -0
  48. model-00045-of-00080.safetensors +3 -0
  49. model-00046-of-00080.safetensors +3 -0
  50. model-00047-of-00080.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+ # Twilight-XL
10
+
11
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
+
13
+ ## Merge Details
14
+ ### Merge Method
15
+
16
+ This model was merged using the passthrough merge method.
17
+
18
+ ### Models Merged
19
+
20
+ The following models were included in the merge:
21
+ * /mnt/sda/ai/models/Twilight-Large
22
+
23
+ ### Configuration
24
+
25
+ The following YAML configuration was used to produce this model:
26
+
27
+ ```yaml
28
+ dtype: bfloat16
29
+ merge_method: passthrough
30
+ slices:
31
+ - sources:
32
+ - layer_range: [0, 70]
33
+ model: /mnt/sda/ai/models/Twilight-Large
34
+ - sources:
35
+ - layer_range: [18, 87]
36
+ model: /mnt/sda/ai/models/Twilight-Large
37
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "softwareweaver/Twilight-XL-195B",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 139,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "bfloat16",
24
+ "transformers_version": "4.45.2",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ dtype: bfloat16
2
+ merge_method: passthrough
3
+ slices:
4
+ - sources:
5
+ - layer_range: [0, 70]
6
+ model: softwareweaver/Twilight-Large-123B
7
+ - sources:
8
+ - layer_range: [18, 87]
9
+ model: softwareweaver/Twilight-Large-123B
model-00001-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:28ea143453695b72170d67935ed31741b16d2367a7a2e2f5e85b98af712bd4d8
3
+ size 4378928504
model-00002-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77c768041acdad2ef31ed8e094b44786d953737460975ba643e98c39731558be
3
+ size 4907411088
model-00003-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c65331712512eb118dbc66c6a02da45811ca1b19c2aca656b1d3de2273559b3
3
+ size 4806747904
model-00004-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c38ee620deeeb46835b592529f41dc087b3ac81b64321ea745fac98f23a711f1
3
+ size 4831938544
model-00005-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9cc0cfc46df3006b2320ad9b015b2cefb55473e725f80e4ae498de1f1df88849
3
+ size 4831938552
model-00006-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cddfe2c102927819db4a0aa35daef91fa0a4179de1ee8feb5960fab69a07dee3
3
+ size 4907411096
model-00007-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c5f84c352b9b0945564869cdb72f3beda638af3f2b340e66981a0065d97a4841
3
+ size 4907435776
model-00008-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:23b5ba653ceca1fcd994350156d10a7d4f1fd6736caf0fbcefb993fcbfe23591
3
+ size 4781557264
model-00009-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e3ecd5a9b54c5fd36592bb693f68f349176db8cdd40987583dc7de5f7c3a2ea
3
+ size 4781656640
model-00010-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:54aeba6c96f42ebee3c6e9b315b825e2e052a92952749c1f8f5cac2db6cee58f
3
+ size 4882220464
model-00011-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:56c3094e1c17640f17953ddca39bb4d8ecc819582e1085c6d2fcc4444acdad1d
3
+ size 4932601720
model-00012-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7d8bac7539a27a0bd3d464fe41367bdab9f190d0e3da6528f15916208fcc705d
3
+ size 4781557264
model-00013-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42cd2834f77f9101f70f4e9f402c9e21342df57935d30c848bc783b09bc4e436
3
+ size 4831938552
model-00014-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d7b88516a12288aff7f4497b3d93b8b99252ffb1e97cc5f1089c5bf7535ffaa
3
+ size 4831938544
model-00015-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:543e69e6f844b32f04d84c20e957feaef52e1abd7e682fd5f85c4116abf7b73b
3
+ size 4831938552
model-00016-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c9b6c5ddabf84e858773dcc2164dd6f3251b3b06d19cd1946c74c0530868d0d7
3
+ size 4831938552
model-00017-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d7e99a93910ce4f7a44310fed31183923847aaf2b0f4cc8595c4ba57a8f9a0f
3
+ size 4831938552
model-00018-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2182154a94e202135934dff296c3ce0a4e05b92d4a3a6e258a72d37470e09c5
3
+ size 4882220464
model-00019-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:07cd6c91f6ded5c8ab76cc709a215eb6d3f2d941e53fe6549fd0bef2cfab7f5d
3
+ size 4932601720
model-00020-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b53e645658c57d5cdfbe91e4c6ce6701455099d9be4f562a7378d91f820cc16f
3
+ size 4781557264
model-00021-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b82ab040d02623d750bd05db82e69de7a4a815c20e18b1a69c984d1f6ae077ef
3
+ size 4781656640
model-00022-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:76778eb96f3e24dffa8e484d83a6d3ad59cdd6bc2d8d7da5727a1a770d622a17
3
+ size 4882220464
model-00023-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:895c7f5c0dd1921365376b1e0b1f261baccdaa93d7d426b6ce0bf97f4f267ad8
3
+ size 4932601720
model-00024-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dd100cf975e561f74b10d98096dde01b8160d4b7eb0c3a0ce7ea54b9a3fd46ab
3
+ size 4781557264
model-00025-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b723f922fc22ca2ab195890d27e1de82f1e0322b69ac6d211bbce91d1ff9fc98
3
+ size 4831938552
model-00026-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab04d708a98e7a37c15514469c59ab007c0a5ee4ed4ba92b8c207b3d450e4493
3
+ size 4831938544
model-00027-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5c1d78564aacc3592b268d281198cce5958479e048136b1a758ede93486387f6
3
+ size 4831938552
model-00028-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2622cb7aba72a584ed3f7f5cbe22697d70a85e4fe727c3faf299819521236ee4
3
+ size 4831938552
model-00029-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b88667b0f68c2b2bc7804072b2cb5a9fd0eaaa153e8ff0b8b0634011c8733075
3
+ size 4831938552
model-00030-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e4ad8d9f361e40ab2a9d2fb3f636ca8b5f0dcdb4549e3e07864e4c7da2cb3a7
3
+ size 4882220464
model-00031-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c41d5bf0b418bec12b9f8ae659063bd7264fe2e777674d85a0945a6492ccdf98
3
+ size 4932601720
model-00032-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:28e95285082b5e31a48cd78070c01145fc9b5ae3f28876e4a898d18bfd21ea99
3
+ size 4781557264
model-00033-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b1b1b1f3c2037a94d1508cde0c1a6e6feeae79ddcff8f6fb5be90bfe09aa9878
3
+ size 4781656640
model-00034-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0a01eb2e1db4f6fde802911820ab22d293363a49ce7194f4f2d72117930fce4e
3
+ size 4882220464
model-00035-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d5cb9562b2d800dc3b5a2065f5dd390bf62c13cfaa808f21f885ed687d2c7f06
3
+ size 4932601720
model-00036-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b8abbcf9585ad121c0b442edb442cb2e0f43fecbd59fbd170fb16415ef3548fe
3
+ size 4781557264
model-00037-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04515c15a6a5ccf5f3b4e33d8a3f20d5d25369f5675a8bb87b9f372ed39640e1
3
+ size 4831938552
model-00038-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59f03202bcf8137db16456c6fdb48fa3a58e7d235b52ac89b7b020dca8237b45
3
+ size 4831938544
model-00039-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:08f6a4c36913b70a2f841581b7f13d07be6ecd1d98616f2a03f3709729b660c8
3
+ size 4831938552
model-00040-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:19b64b23afbb45a6bf775f6f0a4b38e584074799ba1269be34d05b786462d2b5
3
+ size 4831938552
model-00041-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a28d969aa45586aa95e9b8ad5c1080b32db1e77bc9b0fe38fd1f14f52401d777
3
+ size 4831938552
model-00042-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e51f9d77a0d4beffa2799fe25f326128312cba1156f3428e5cfbf79214306cc9
3
+ size 4882220464
model-00043-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d889937968bf1d752e961f013b67082715adb05a232f84519fc60f77dfb47877
3
+ size 4932601720
model-00044-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3c067cf95ab6b519d943d4cd1469ff6d03e3a442d12e5bb741a1753b0a12bf6c
3
+ size 4781557272
model-00045-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84907d1b8a4ea38e9e18accc03a8736c21a04c6a03cbcf6e156d346768907f68
3
+ size 4781656632
model-00046-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:663842ad0b82385b2bd96b2555799aa55ba0a3ec28e99a2a9610da025e2164a6
3
+ size 4882220464
model-00047-of-00080.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fb2273367496b0c3ca62827e8795029c01bd890d1882a138c5d987926735234
3
+ size 4932601728