Update README.md

README.md

@@ -30,6 +30,8 @@ license: apache-2.0
 2、RAG优化，适配更多真实业务场景；
 3、适配长文本做rerank。
 
+![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/64745e955aba8edfb2ed561a/NyV_6ZrsaqUluUnxHKR_m.jpeg)
+
 -----------------------------------------
 <details open="open">
 <summary>Click to Open Contents</summary>
@@ -40,7 +42,8 @@ license: apache-2.0
 - <a href="#-model-list" target="_Self">🍎 Model List</a>
 - <a href="#-manual" target="_Self">📖 Manual</a>
   - <a href="#installation" target="_Self">Installation</a>
-  - <a href="#quick-start" target="_Self">Quick Start</a>
+  - <a href="#quick-start" target="_Self">Quick Start (`transformers`, `sentence-transformers`)</a>
+  - <a href="#integrations-for-rag-frameworks" target="_Self">Integrations for RAG Frameworks (`langchain`, `llama_index`)</a>
 - <a href="#%EF%B8%8F-evaluation" target="_Self">⚙️ Evaluation</a>
   - <a href="#evaluate-semantic-representation-by-mteb" target="_Self">Evaluate Semantic Representation by MTEB</a>
   - <a href="#evaluate-rag-by-llamaindex" target="_Self">Evaluate RAG by LlamaIndex</a>
@@ -134,17 +137,20 @@ Existing embedding models often encounter performance challenges in bilingual an
 ### Installation
 
 First, create a conda environment and activate it.
+
 ```bash
 conda create --name bce python=3.10 -y
 conda activate bce
 ```
 
-Then install `BCEmbedding`:
+Then install `BCEmbedding` for minimal installation:
+
 ```bash
-pip install git+https://github.com/netease-youdao/BCEmbedding.git
+pip install BCEmbedding==0.1.1
 ```
 
 Or install from source:
+
 ```bash
 git clone [email protected]:netease-youdao/BCEmbedding.git
 cd BCEmbedding
@@ -153,7 +159,9 @@ pip install -v -e .
 
 ### Quick Start
 
-Use `EmbeddingModel` by `BCEmbedding`, and `cls` [pooler](https://github.com/netease-youdao/BCEmbedding/blob/master/BCEmbedding/models/embedding.py#L24) is default.
+#### 1. Based on `BCEmbedding`
+
+Use `EmbeddingModel`, and `cls` [pooler](./BCEmbedding/models/embedding.py#L24) is default.
 
 ```python
 from BCEmbedding import EmbeddingModel
@@ -168,7 +176,7 @@ model = EmbeddingModel(model_name_or_path="maidalun1020/bce-embedding-base_v1")
 embeddings = model.encode(sentences)
 ```
 
-Use `RerankerModel` by `BCEmbedding` to calculate relevant scores and rerank:
+Use `RerankerModel` to calculate relevant scores and rerank:
 
 ```python
 from BCEmbedding import RerankerModel
@@ -190,6 +198,164 @@ scores = model.compute_score(sentence_pairs)
 rerank_results = model.rerank(query, passages)
 ```
 
+NOTE:
+
+- In [`RerankerModel.rerank`](./BCEmbedding/models/reranker.py#L137) method, we provide an advanced preproccess that we use in production for making `sentence_pairs`, when "passages" are very long.
+
+#### 2. Based on `transformers`
+
+For `EmbeddingModel`:
+
+```python
+from transformers import AutoModel, AutoTokenizer
+
+# list of sentences
+sentences = ['sentence_0', 'sentence_1', ...]
+
+# init model and tokenizer
+tokenizer = AutoTokenizer.from_pretrained('maidalun1020/bce-embedding-base_v1')
+model = AutoModel.from_pretrained('maidalun1020/bce-embedding-base_v1')
+
+device = 'cuda'  # if no GPU, set "cpu"
+model.to(device)
+
+# get inputs
+inputs = tokenizer(sentences, padding=True, truncation=True, max_length=512, return_tensors="pt")
+inputs_on_device = {k: v.to(self.device) for k, v in inputs.items()}
+
+# get embeddings
+outputs = model(**inputs_on_device, return_dict=True)
+embeddings = outputs.last_hidden_state[:, 0]  # cls pooler
+embeddings = embeddings / embeddings.norm(dim=1, keepdim=True)  # normalize
+```
+
+For `RerankerModel`:
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
+# init model and tokenizer
+tokenizer = AutoTokenizer.from_pretrained('maidalun1020/bce-reranker-base_v1')
+model = AutoModelForSequenceClassification.from_pretrained('maidalun1020/bce-reranker-base_v1')
+
+device = 'cuda'  # if no GPU, set "cpu"
+model.to(device)
+
+# get inputs
+inputs = tokenizer(sentence_pairs, padding=True, truncation=True, max_length=512, return_tensors="pt")
+inputs_on_device = {k: v.to(device) for k, v in inputs.items()}
+
+# calculate scores
+scores = model(**inputs_on_device, return_dict=True).logits.view(-1,).float()
+scores = torch.sigmoid(scores)
+```
+
+#### 3. Based on `sentence_transformers`
+
+For `EmbeddingModel`:
+
+```python
+from sentence_transformers import SentenceTransformer
+
+# list of sentences
+sentences = ['sentence_0', 'sentence_1', ...]
+
+# init embedding model
+## New update for sentence-trnasformers. So clean up your "`SENTENCE_TRANSFORMERS_HOME`/maidalun1020_bce-embedding-base_v1" or "～/.cache/torch/sentence_transformers/maidalun1020_bce-embedding-base_v1" first for downloading new version.
+model = SentenceTransformer("maidalun1020/bce-embedding-base_v1")
+
+# extract embeddings
+embeddings = model.encode(sentences, normalize_embeddings=True)
+```
+
+For `RerankerModel`:
+
+```python
+from sentence_transformers import CrossEncoder
+
+# init reranker model
+model = CrossEncoder('maidalun1020/bce-reranker-base_v1', max_length=512)
+
+# calculate scores of sentence pairs
+scores = model.predict(sentence_pairs)
+```
+
+### Integrations for RAG Frameworks
+
+#### 1. Used in `langchain`
+
+```python
+from langchain.embeddings import HuggingFaceEmbeddings
+from langchain_community.vectorstores import FAISS
+from langchain_community.vectorstores.utils import DistanceStrategy
+
+query = 'apples'
+passages = [
+        'I like apples', 
+        'I like oranges', 
+        'Apples and oranges are fruits'
+    ]
+  
+# init embedding model
+model_name = 'maidalun1020/bce-embedding-base_v1'
+model_kwargs = {'device': 'cuda'}
+encode_kwargs = {'batch_size': 64, 'normalize_embeddings': True, 'show_progress_bar': False}
+
+embed_model = HuggingFaceEmbeddings(
+    model_name=model_name,
+    model_kwargs=model_kwargs,
+    encode_kwargs=encode_kwargs
+  )
+
+# example #1. extract embeddings
+query_embedding = embed_model.embed_query(query)
+passages_embeddings = embed_model.embed_documents(passages)
+
+# example #2. langchain retriever example
+faiss_vectorstore = FAISS.from_texts(passages, embed_model, distance_strategy=DistanceStrategy.MAX_INNER_PRODUCT)
+
+retriever = faiss_vectorstore.as_retriever(search_type="similarity", search_kwargs={"score_threshold": 0.5, "k": 3})
+
+related_passages = retriever.get_relevant_documents(query)
+```
+
+#### 2. Used in `llama_index`
+
+```python
+from llama_index.embeddings import HuggingFaceEmbedding
+from llama_index import VectorStoreIndex, ServiceContext, SimpleDirectoryReader
+from llama_index.node_parser import SimpleNodeParser
+from llama_index.llms import OpenAI
+
+query = 'apples'
+passages = [
+        'I like apples', 
+        'I like oranges', 
+        'Apples and oranges are fruits'
+    ]
+
+# init embedding model
+model_args = {'model_name': 'maidalun1020/bce-embedding-base_v1', 'max_length': 512, 'embed_batch_size': 64, 'device': 'cuda'}
+embed_model = HuggingFaceEmbedding(**model_args)
+
+# example #1. extract embeddings
+query_embedding = embed_model.get_query_embedding(query)
+passages_embeddings = embed_model.get_text_embedding_batch(passages)
+
+# example #2. rag example
+llm = OpenAI(model='gpt-3.5-turbo-0613', api_key=os.environ.get('OPENAI_API_KEY'), api_base=os.environ.get('OPENAI_BASE_URL'))
+service_context = ServiceContext.from_defaults(llm=llm, embed_model=embed_model)
+
+documents = SimpleDirectoryReader(input_files=["BCEmbedding/tools/eval_rag/eval_pdfs/Comp_en_llama2.pdf"]).load_data()
+node_parser = SimpleNodeParser.from_defaults(chunk_size=512)
+nodes = node_parser.get_nodes_from_documents(documents[0:36])
+index = VectorStoreIndex(nodes, service_context=service_context)
+query_engine = index.as_query_engine()
+response = query_engine.query("What is llama?")
+```
+
+
 ## ⚙️ Evaluation
 
 ### Evaluate Semantic Representation by MTEB
@@ -200,9 +366,9 @@ We provide evaluateion tools for `embedding` and `reranker` models, based on [MT
 
 #### 1. Embedding Models
 
-Just run following cmd to evaluate `your_embedding_model` (e.g. `maidalun1020/bce-embedding-base_v1`) in **monolingual, bilingual and crosslingual settings** (e.g. `["en", "zh", "en-zh", "zh-en"]`).
+Just run following cmd to evaluate `your_embedding_model` (e.g. `maidalun1020/bce-embedding-base_v1`) in **bilingual and crosslingual settings** (e.g. `["en", "zh", "en-zh", "zh-en"]`).
 
-  运行下面命令评测`your_embedding_model`（比如，`maidalun1020/bce-embedding-base_v1`）。评测任务将会在**单语种，双语种和跨语种**（比如，`["en", "zh", "en-zh", "zh-en"]`）模式下评测：
+  运行下面命令评测`your_embedding_model`（比如，`maidalun1020/bce-embedding-base_v1`）。评测任务将会在**双语和跨语种**（比如，`["en", "zh", "en-zh", "zh-en"]`）模式下评测：
 
 ```bash
 python BCEmbedding/tools/eval_mteb/eval_embedding_mteb.py --model_name_or_path maidalun1020/bce-embedding-base_v1 --pooler cls
@@ -213,8 +379,11 @@ The total evaluation tasks contain ***114 datastes*** of **"Retrieval", "STS", "
   评测包含 **"Retrieval"， "STS"， "PairClassification"， "Classification"， "Reranking"和"Clustering"** 这六大类任务的 ***114个数据集***。
 
 ***NOTE:***
-- All models are evaluated in their **recommended pooling method (`pooler`)**. "jina-embeddings-v2-base-en", "m3e-base" and "m3e-large" use `mean` pooler, while the others use `cls`.
+- **All models are evaluated in their recommended pooling method (`pooler`)**.
+  - `mean` pooler: "jina-embeddings-v2-base-en", "m3e-base", "m3e-large", "e5-large-v2", "multilingual-e5-base", "multilingual-e5-large" and "gte-large".
+  - `cls` pooler: Other models.
 - "jina-embeddings-v2-base-en" model should be loaded with `trust_remote_code`.
+
 ```bash
 python BCEmbedding/tools/eval_mteb/eval_embedding_mteb.py --model_name_or_path {moka-ai/m3e-base | moka-ai/m3e-large} --pooler mean
 
@@ -222,14 +391,14 @@ python BCEmbedding/tools/eval_mteb/eval_embedding_mteb.py --model_name_or_path j
 ```
 
   ***注意：***
-  - 所有模型的评测采用各自推荐的`pooler`。"jina-embeddings-v2-base-en"， "m3e-base"和"m3e-large"的 `pooler`采用`mean`，其他模型的`pooler`采用`cls`.
+  - 所有模型的评测采用各自推荐的`pooler`。"jina-embeddings-v2-base-en", "m3e-base", "m3e-large", "e5-large-v2", "multilingual-e5-base", "multilingual-e5-large"和"gte-large"的 `pooler`采用`mean`，其他模型的`pooler`采用`cls`.
   - "jina-embeddings-v2-base-en"模型在载入时需要`trust_remote_code`。
 
 #### 2. Reranker Models
 
-Run following cmd to evaluate `your_reranker_model` (e.g. "maidalun1020/bce-reranker-base_v1") in **monolingual, bilingual and crosslingual settings** (e.g. `["en", "zh", "en-zh", "zh-en"]`).
+Run following cmd to evaluate `your_reranker_model` (e.g. "maidalun1020/bce-reranker-base_v1") in **bilingual and crosslingual settings** (e.g. `["en", "zh", "en-zh", "zh-en"]`).
 
-  运行下面命令评测`your_reranker_model`（比如，`maidalun1020/bce-reranker-base_v1`）。评测任务将会在**单语种，双语种和跨语种**（比如，`["en", "zh", "en-zh", "zh-en"]`）模式下评测：
+  运行下面命令评测`your_reranker_model`（比如，`maidalun1020/bce-reranker-base_v1`）。评测任务将会在 **双语种和跨语种**（比如，`["en", "zh", "en-zh", "zh-en"]`）模式下评测：
 
 ```bash
 python BCEmbedding/tools/eval_mteb/eval_reranker_mteb.py --model_name_or_path maidalun1020/bce-reranker-base_v1
@@ -347,13 +516,13 @@ The summary of multiple domains evaluations can be seen in <a href=#1-multiple-d
 | ***bce-embedding-base_v1*** | 768 | `cls` | Free | 57.60 | 65.73 | 74.96 | 69.00 | 57.29 | 38.95 | 59.43 |  
 
 ***NOTE:***
-- Our ***bce-embedding-base_v1*** outperforms other opensource embedding models with various model size.
+- Our ***bce-embedding-base_v1*** outperforms other opensource embedding models with comparable model size.
 - ***114 datastes*** of **"Retrieval", "STS", "PairClassification", "Classification", "Reranking" and "Clustering"** in `["en", "zh", "en-zh", "zh-en"]` setting.
 - The [crosslingual evaluation datasets](https://github.com/netease-youdao/BCEmbedding/blob/master/BCEmbedding/evaluation/c_mteb/Retrieval.py) we released belong to `Retrieval` task.
 - More evaluation details please check [Embedding Models Evaluation Summary](https://github.com/netease-youdao/BCEmbedding/blob/master/Docs/EvaluationSummary/embedding_eval_summary.md).
 
   ***要点：***
-  - 对比所有开源的各种规模的embedding模型，***bce-embedding-base_v1*** 表现最好。
+  - 对比其他开源的相同规模的embedding模型，***bce-embedding-base_v1*** 表现最好，效果比最好的large模型稍差。
   - 评测包含 **"Retrieval"， "STS"， "PairClassification"， "Classification"， "Reranking"和"Clustering"** 这六大类任务的共 ***114个数据集***。
   - 我们开源的[跨语种语义表征评测数据](https://github.com/netease-youdao/BCEmbedding/blob/master/BCEmbedding/evaluation/c_mteb/Retrieval.py)属于`Retrieval`任务。
   - 更详细的评测结果详见[Embedding模型指标汇总](https://github.com/netease-youdao/BCEmbedding/blob/master/Docs/EvaluationSummary/embedding_eval_summary.md)。