|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
pipeline_tag: sentence-similarity |
|
--- |
|
Repository with files to perform BM25 searches with [FastEmbed](https://github.com/qdrant/fastembed). |
|
|
|
[BM25 (Best Matching 25)](https://en.wikipedia.org/wiki/Okapi_BM25) is a ranking function used by search engines to estimate the relevance of documents to a given search query. |
|
|
|
### Usage |
|
|
|
> Note: |
|
This model is supposed to be used with Qdrant. Vectors have to be configured with [Modifier.IDF](https://qdrant.tech/documentation/concepts/indexing/?q=modifier#idf-modifier). |
|
|
|
Here's an example of BM25 with [FastEmbed](https://github.com/qdrant/fastembed). |
|
|
|
```py |
|
from fastembed import SparseTextEmbedding |
|
|
|
documents = [ |
|
"You should stay, study and sprint.", |
|
"History can only prepare us to be surprised yet again.", |
|
] |
|
|
|
model = SparseTextEmbedding(model_name="Qdrant/bm25") |
|
embeddings = list(model.embed(documents)) |
|
|
|
# [ |
|
# SparseEmbedding( |
|
# values=array([1.67419738, 1.67419738, 1.67419738, 1.67419738]), |
|
# indices=array([171321964, 1881538586, 150760872, 1932363795])), |
|
# SparseEmbedding(values=array( |
|
# [1.66973021, 1.66973021, 1.66973021, 1.66973021, 1.66973021]), |
|
# indices=array([ |
|
# 578407224, 1849833631, 1008800696, 2090661150, |
|
# 1117393019 |
|
# ])) |
|
# ] |
|
``` |
|
|
|
|
|
|
|
``` |
