GGUF
English
TensorBlock
GGUF
Eval Results
Inference Endpoints
morriszms commited on
Commit
bec92b9
·
verified ·
1 Parent(s): 4f29675

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ falcon-mamba-7b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ falcon-mamba-7b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ falcon-mamba-7b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ falcon-mamba-7b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ falcon-mamba-7b-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ falcon-mamba-7b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ falcon-mamba-7b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ falcon-mamba-7b-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ falcon-mamba-7b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ falcon-mamba-7b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ falcon-mamba-7b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ falcon-mamba-7b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,172 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ datasets:
5
+ - tiiuae/falcon-refinedweb
6
+ - HuggingFaceFW/fineweb-edu
7
+ license: other
8
+ license_name: falcon-mamba-7b-license
9
+ license_link: https://falconllm.tii.ae/falcon-mamba-7b-terms-and-conditions.html
10
+ base_model: tiiuae/falcon-mamba-7b
11
+ tags:
12
+ - TensorBlock
13
+ - GGUF
14
+ model-index:
15
+ - name: falcon-mamba-7b
16
+ results:
17
+ - task:
18
+ type: text-generation
19
+ name: Text Generation
20
+ dataset:
21
+ name: IFEval (0-Shot)
22
+ type: HuggingFaceH4/ifeval
23
+ args:
24
+ num_few_shot: 0
25
+ metrics:
26
+ - type: inst_level_strict_acc and prompt_level_strict_acc
27
+ value: 33.36
28
+ name: strict accuracy
29
+ source:
30
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/falcon-mamba-7b
31
+ name: Open LLM Leaderboard
32
+ - task:
33
+ type: text-generation
34
+ name: Text Generation
35
+ dataset:
36
+ name: BBH (3-Shot)
37
+ type: BBH
38
+ args:
39
+ num_few_shot: 3
40
+ metrics:
41
+ - type: acc_norm
42
+ value: 19.88
43
+ name: normalized accuracy
44
+ source:
45
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/falcon-mamba-7b
46
+ name: Open LLM Leaderboard
47
+ - task:
48
+ type: text-generation
49
+ name: Text Generation
50
+ dataset:
51
+ name: MATH Lvl 5 (4-Shot)
52
+ type: hendrycks/competition_math
53
+ args:
54
+ num_few_shot: 4
55
+ metrics:
56
+ - type: exact_match
57
+ value: 3.63
58
+ name: exact match
59
+ source:
60
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/falcon-mamba-7b
61
+ name: Open LLM Leaderboard
62
+ - task:
63
+ type: text-generation
64
+ name: Text Generation
65
+ dataset:
66
+ name: GPQA (0-shot)
67
+ type: Idavidrein/gpqa
68
+ args:
69
+ num_few_shot: 0
70
+ metrics:
71
+ - type: acc_norm
72
+ value: 8.05
73
+ name: acc_norm
74
+ source:
75
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/falcon-mamba-7b
76
+ name: Open LLM Leaderboard
77
+ - task:
78
+ type: text-generation
79
+ name: Text Generation
80
+ dataset:
81
+ name: MuSR (0-shot)
82
+ type: TAUR-Lab/MuSR
83
+ args:
84
+ num_few_shot: 0
85
+ metrics:
86
+ - type: acc_norm
87
+ value: 10.86
88
+ name: acc_norm
89
+ source:
90
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/falcon-mamba-7b
91
+ name: Open LLM Leaderboard
92
+ - task:
93
+ type: text-generation
94
+ name: Text Generation
95
+ dataset:
96
+ name: MMLU-PRO (5-shot)
97
+ type: TIGER-Lab/MMLU-Pro
98
+ config: main
99
+ split: test
100
+ args:
101
+ num_few_shot: 5
102
+ metrics:
103
+ - type: acc
104
+ value: 14.47
105
+ name: accuracy
106
+ source:
107
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=tiiuae/falcon-mamba-7b
108
+ name: Open LLM Leaderboard
109
+ ---
110
+
111
+ <div style="width: auto; margin-left: auto; margin-right: auto">
112
+ <img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">
113
+ </div>
114
+ <div style="display: flex; justify-content: space-between; width: 100%;">
115
+ <div style="display: flex; flex-direction: column; align-items: flex-start;">
116
+ <p style="margin-top: 0.5em; margin-bottom: 0em;">
117
+ Feedback and support: TensorBlock's <a href="https://x.com/tensorblock_aoi">Twitter/X</a>, <a href="https://t.me/TensorBlock">Telegram Group</a> and <a href="https://x.com/tensorblock_aoi">Discord server</a>
118
+ </p>
119
+ </div>
120
+ </div>
121
+
122
+ ## tiiuae/falcon-mamba-7b - GGUF
123
+
124
+ This repo contains GGUF format model files for [tiiuae/falcon-mamba-7b](https://huggingface.co/tiiuae/falcon-mamba-7b).
125
+
126
+ The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
127
+
128
+ ## Prompt template
129
+
130
+ ```
131
+
132
+ ```
133
+
134
+ ## Model file specification
135
+
136
+ | Filename | Quant type | File Size | Description |
137
+ | -------- | ---------- | --------- | ----------- |
138
+ | [falcon-mamba-7b-Q2_K.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q2_K.gguf) | Q2_K | 2.389 GB | smallest, significant quality loss - not recommended for most purposes |
139
+ | [falcon-mamba-7b-Q3_K_S.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q3_K_S.gguf) | Q3_K_S | 3.050 GB | very small, high quality loss |
140
+ | [falcon-mamba-7b-Q3_K_M.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q3_K_M.gguf) | Q3_K_M | 3.050 GB | very small, high quality loss |
141
+ | [falcon-mamba-7b-Q3_K_L.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q3_K_L.gguf) | Q3_K_L | 3.050 GB | small, substantial quality loss |
142
+ | [falcon-mamba-7b-Q4_0.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q4_0.gguf) | Q4_0 | 3.915 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
143
+ | [falcon-mamba-7b-Q4_K_S.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q4_K_S.gguf) | Q4_K_S | 3.915 GB | small, greater quality loss |
144
+ | [falcon-mamba-7b-Q4_K_M.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q4_K_M.gguf) | Q4_K_M | 3.915 GB | medium, balanced quality - recommended |
145
+ | [falcon-mamba-7b-Q5_0.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q5_0.gguf) | Q5_0 | 4.730 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
146
+ | [falcon-mamba-7b-Q5_K_S.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q5_K_S.gguf) | Q5_K_S | 4.730 GB | large, low quality loss - recommended |
147
+ | [falcon-mamba-7b-Q5_K_M.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q5_K_M.gguf) | Q5_K_M | 4.730 GB | large, very low quality loss - recommended |
148
+ | [falcon-mamba-7b-Q6_K.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q6_K.gguf) | Q6_K | 5.595 GB | very large, extremely low quality loss |
149
+ | [falcon-mamba-7b-Q8_0.gguf](https://huggingface.co/tensorblock/falcon-mamba-7b-GGUF/tree/main/falcon-mamba-7b-Q8_0.gguf) | Q8_0 | 7.232 GB | very large, extremely low quality loss - not recommended |
150
+
151
+
152
+ ## Downloading instruction
153
+
154
+ ### Command line
155
+
156
+ Firstly, install Huggingface Client
157
+
158
+ ```shell
159
+ pip install -U "huggingface_hub[cli]"
160
+ ```
161
+
162
+ Then, downoad the individual model file the a local directory
163
+
164
+ ```shell
165
+ huggingface-cli download tensorblock/falcon-mamba-7b-GGUF --include "falcon-mamba-7b-Q2_K.gguf" --local-dir MY_LOCAL_DIR
166
+ ```
167
+
168
+ If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:
169
+
170
+ ```shell
171
+ huggingface-cli download tensorblock/falcon-mamba-7b-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'
172
+ ```
falcon-mamba-7b-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8325c406eda952b24c46d08c792b755486c2e1e3d7809398e34e1aabe0f4635
3
+ size 2565011200
falcon-mamba-7b-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cfb9598415e196bd3f7a007a566ecda6a6c21423fb3947f05ed5fe48eb723044
3
+ size 3275339520
falcon-mamba-7b-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:611ddf623476068e97afadd9faa6a7030f065464d70d19746f62b5f6c61ca169
3
+ size 3275339520
falcon-mamba-7b-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1566471de457162ef3c9c379fb91bfc3a5880b4a41831741f16f8dabeaed6c4e
3
+ size 3275339520
falcon-mamba-7b-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88e6c41a1eb366f1405da691d41283b43b6c99b0507f750bf8a443fd24d7d1c1
3
+ size 4204230400
falcon-mamba-7b-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eaa31dbf8058c867844df9a47f1a4834dfc413e5f9d0ce10d710904668bede7f
3
+ size 4204230400
falcon-mamba-7b-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3b4c9fd92701725cb511bd5b6a94a38e0e03baeae51c1212ee8254ea44b6880
3
+ size 4204230400
falcon-mamba-7b-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f70d60c9b92cf1e78f5aa31fdded73992461724d17470e5f2bf67b8a81ff2996
3
+ size 5078480640
falcon-mamba-7b-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c032f66d7c42362872b8e69789d7f8d15f89b46f1b22097bb125f7e2af5edc67
3
+ size 5078480640
falcon-mamba-7b-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b991276af7a5a799b17b39593373afaa75b27b685423321a87ac7459b5618d45
3
+ size 5078480640
falcon-mamba-7b-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e7a14c58e487988aadaa1c1d078b1d41327ba89c3a3e3d54f890af65cfc8898f
3
+ size 6007371520
falcon-mamba-7b-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e2415f0eab5cc36d3a1c930e8235448311376c18e344bda5893f78059c80f1dc
3
+ size 7765735168