File size: 9,560 Bytes
0716ff1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 |
---
library_name: transformers
license: apache-2.0
language:
- en
- zh
- es
- de
- ar
- ru
- ja
- ko
- hi
- sk
- vi
- tr
- fi
- id
- fa
- 'no'
- th
- sv
- pt
- da
- bn
- te
- ro
- it
- fr
- nl
- sw
- pl
- hu
- cs
- el
- uk
- mr
- ta
- tl
- bg
- lt
- ur
- he
- gu
- kn
- am
- kk
- hr
- uz
- jv
- ca
- az
- ms
- sr
- sl
- yo
- lv
- is
- ha
- ka
- et
- bs
- hy
- ml
- pa
- mt
- km
- sq
- or
- as
- my
- mn
- af
- be
- ga
- mk
- cy
- gl
- ceb
- la
- yi
- lb
- tg
- gd
- ne
- ps
- eu
- ky
- ku
- si
- ht
- eo
- lo
- fy
- sd
- mg
- so
- ckb
- su
- nn
datasets:
- lightblue/reranker_continuous_filt_max7_train
base_model:
- Qwen/Qwen2.5-0.5B-Instruct
pipeline_tag: text-generation
tags:
- reranker
---
[](https://hf.co/QuantFactory)
# QuantFactory/lb-reranker-0.5B-v1.0-GGUF
This is quantized version of [lightblue/lb-reranker-0.5B-v1.0](https://huggingface.co/lightblue/lb-reranker-0.5B-v1.0) created using llama.cpp
# Original Model Card
# LB Reranker v1.0
<div style="width: 100%; height: 160px;
display: flex; align-items: center;
justify-content: center;
border: 8px solid black;
font-size: 120px; font-weight: bold;
text-align: center;
color: #438db8;
font-family: 'Helvetica Neue', sans-serif;">
LBR
</div>
The LB Reranker has been trained to determine the relatedness of a given query to a piece of text, therefore allowing it to be used as a ranker or reranker in various retrieval-based tasks.
This model is fine-tuned from a [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) model checkpoint and was trained for roughly 5.5 hours using the 8 x L20 instance ([ecs.gn8is-8x.32xlarge](https://www.alibabacloud.com/help/en/ecs/user-guide/gpu-accelerated-compute-optimized-and-vgpu-accelerated-instance-families-1)) on [Alibaba Cloud](https://www.alibabacloud.com/).
The training data for this model can be found at [lightblue/reranker_continuous_filt_max7_train](https://huggingface.co/datasets/lightblue/reranker_continuous_filt_max7_train) and the code for generating this data as well as running the training of the model can be found on [our Github repo](https://github.com/lightblue-tech/lb-reranker).
Trained on data in over 95 languages, this model is applicable to a broad range of use cases.
This model has three main benefits over comparable rerankers.
1. It has shown slightly higher performance on evaluation benchmarks.
2. It has been trained on more languages than any previous model.
3. It is a simple Causal LM model trained to output a string between "1" and "7".
This last point means that this model can be used natively with many widely available inference packages, including vLLM and LMDeploy.
This in turns allows our reranker to benefit from improvements to inference as and when these packages release them.
Update: We have also found that this model works pretty well as a code snippet reranker too (P@1 of 96%)! See our [Colab](https://colab.research.google.com/drive/1ABL1xaarekLIlVJKbniYhXgYu6ZNwfBm?usp=sharing) for more details.
# How to use
The model was trained to expect an input such as:
```
<<<Query>>>
{your_query_here}
<<<Context>>>
{your_context_here}
```
And to output a string of a number between 1-7.
In order to make a continuous score that can be used for reranking query-context pairs (i.e. a method with few ties), we calculate the expectation value of the scores.
We include scripts to do this in both vLLM and LMDeploy:
#### vLLM
Install [vLLM](https://github.com/vllm-project/vllm/) using `pip install vllm`.
```python
from vllm import LLM, SamplingParams
import numpy as np
def make_reranker_input(t, q):
return f"<<<Query>>>\n{q}\n\n<<<Context>>>\n{t}"
def make_reranker_training_datum(context, question):
system_message = "Given a query and a piece of text, output a score of 1-7 based on how related the query is to the text. 1 means least related and 7 is most related."
return [
{"role": "system", "content": system_message},
{"role": "user", "content": make_reranker_input(context, question)},
]
def get_prob(logprob_dict, tok_id):
return np.exp(logprob_dict[tok_id].logprob) if tok_id in logprob_dict.keys() else 0
llm = LLM("lightblue/lb-reranker-v1.0")
sampling_params = SamplingParams(temperature=0.0, logprobs=14, max_tokens=1)
tok = llm.llm_engine.tokenizer.tokenizer
idx_tokens = [tok.encode(str(i))[0] for i in range(1, 8)]
query_texts = [
("What is the scientific name of apples?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
("What is the Chinese word for 'apple'?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
("What is the square root of 999?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
]
chats = [make_reranker_training_datum(c, q) for q, c in query_texts]
responses = llm.chat(chats, sampling_params)
probs = np.array([[get_prob(r.outputs[0].logprobs[0], y) for y in idx_tokens] for r in responses])
N = probs.shape[1]
M = probs.shape[0]
idxs = np.tile(np.arange(1, N + 1), M).reshape(M, N)
expected_vals = (probs * idxs).sum(axis=1)
print(expected_vals)
# [6.66570732 1.86686378 1.01102923]
```
#### LMDeploy
Install [LMDeploy](https://github.com/InternLM/lmdeploy) using `pip install lmdeploy`.
```python
# Un-comment this if running in a Jupyter notebook, Colab etc.
# import nest_asyncio
# nest_asyncio.apply()
from lmdeploy import GenerationConfig, ChatTemplateConfig, pipeline
import numpy as np
def make_reranker_input(t, q):
return f"<<<Query>>>\n{q}\n\n<<<Context>>>\n{t}"
def make_reranker_training_datum(context, question):
system_message = "Given a query and a piece of text, output a score of 1-7 based on how related the query is to the text. 1 means least related and 7 is most related."
return [
{"role": "system", "content": system_message},
{"role": "user", "content": make_reranker_input(context, question)},
]
def get_prob(logprob_dict, tok_id):
return np.exp(logprob_dict[tok_id]) if tok_id in logprob_dict.keys() else 0
pipe = pipeline(
"lightblue/lb-reranker-v1.0",
chat_template_config=ChatTemplateConfig(
model_name='qwen2d5',
capability='chat'
)
)
tok = pipe.tokenizer.model
idx_tokens = [tok.encode(str(i))[0] for i in range(1, 8)]
query_texts = [
("What is the scientific name of apples?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
("What is the Chinese word for 'apple'?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
("What is the square root of 999?", "An apple is a round, edible fruit produced by an apple tree (Malus spp., among them the domestic or orchard apple; Malus domestica)."),
]
chats = [make_reranker_training_datum(c, q) for q, c in query_texts]
responses = pipe(
chats,
gen_config=GenerationConfig(temperature=1.0, logprobs=14, max_new_tokens=1, do_sample=True)
)
probs = np.array([[get_prob(r.logprobs[0], y) for y in idx_tokens] for r in responses])
N = probs.shape[1]
M = probs.shape[0]
idxs = np.tile(np.arange(1, N + 1), M).reshape(M, N)
expected_vals = (probs * idxs).sum(axis=1)
print(expected_vals)
# [6.66415229 1.84342025 1.01133205]
```
# Evaluation
We perform an evaluation on 9 datasets from the [BEIR benchmark](https://github.com/beir-cellar/beir) that none of the evaluated models have been trained upon (to our knowledge).
* Arguana
* Dbpedia-entity
* Fiqa
* NFcorpus
* Scidocs
* Scifact
* Trec-covid-v2
* Vihealthqa
* Webis-touche2020
We evaluate on a subset of all queries (the first 250) to save evaluation time.
We find that our model performs similarly or better than many of the state-of-the-art reranker models in our evaluation, without compromising on inference speed.
We make our evaluation code and results available [on our Github](https://github.com/lightblue-tech/lb-reranker/blob/main/run_bier.ipynb).
As we can see, this reranker attains greater IR evaluation metrics compared to the two benchmarks we include for all positions apart from @1.
We also show that our model is, on average, faster than the BGE reranker v2.
# License
We share this model under an Apache 2.0 license.
# Developed by
<a href="https://www.lightblue-tech.com">
<img src="https://www.lightblue-tech.com/wp-content/uploads/2023/08/color_%E6%A8%AA%E5%9E%8B-1536x469.png" alt="Lightblue technology logo" width="400"/>
</a>
This model was trained by Peter Devine ([ptrdvn](https://huggingface.co/ptrdvn)) for Lightblue
|