michaelfeil commited on
Commit
2061b8b
·
verified ·
1 Parent(s): 679199c

Update usage with infinity

Browse files

Text-1.5 is currently one of the most deployed models on infinity. I would therefore like to add a note to the Readme on how to use!

```
INFO: Started server process [1]
INFO: Waiting for application startup.
INFO 2024-11-13 00:41:05,291 infinity_emb INFO: infinity_server.py:89
Creating 1engines:
engines=['nomic-ai/nomic-embed-text-v1.5']
INFO 2024-11-13 00:41:05,295 infinity_emb INFO: Anonymized telemetry.py:30
telemetry can be disabled via environment variable
`DO_NOT_TRACK=1`.
INFO 2024-11-13 00:41:05,303 infinity_emb INFO: select_model.py:64
model=`nomic-ai/nomic-embed-text-v1.5` selected,
using engine=`torch` and device=`cuda`
INFO 2024-11-13 00:41:05,489 SentenceTransformer.py:216
sentence_transformers.SentenceTransformer
INFO: Load pretrained SentenceTransformer:
nomic-ai/nomic-embed-text-v1.5
A new version of the following files was downloaded from https://huggingface.co/nomic-ai/nomic-bert-2048:
- configuration_hf_nomic_bert.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
A new version of the following files was downloaded from https://huggingface.co/nomic-ai/nomic-bert-2048:
- modeling_hf_nomic_bert.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
WARNING 2024-11-13 00:41:28,547 modeling_hf_nomic_bert.py:443
transformers_modules.nomic-ai.nomic-bert-
2048.c1b1fd7a715b8eb2e232d34593154ac782c9
8ac9.modeling_hf_nomic_bert WARNING: <All
keys matched successfully>
INFO 2024-11-13 00:41:31,438 infinity_emb INFO: Getting select_model.py:97
timings for batch_size=8 and avg tokens per
sentence=1
0.93 ms tokenization
7.62 ms inference
0.09 ms post-processing
8.64 ms total
embeddings/sec: 926.20
INFO 2024-11-13 00:41:31,504 infinity_emb INFO: Getting select_model.py:103
timings for batch_size=8 and avg tokens per
sentence=512
6.18 ms tokenization
14.68 ms inference
0.11 ms post-processing
20.97 ms total
embeddings/sec: 381.43
```

Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -2786,6 +2786,16 @@ print(output)
2786
  For more information, see the [API reference](https://docs.nomic.ai/reference/endpoints/nomic-embed-text)
2787
 
2788
 
 
 
 
 
 
 
 
 
 
 
2789
  ## Adjusting Dimensionality
2790
 
2791
  `nomic-embed-text-v1.5` is an improvement upon [Nomic Embed](https://huggingface.co/nomic-ai/nomic-embed-text-v1) that utilizes [Matryoshka Representation Learning](https://arxiv.org/abs/2205.13147) which gives developers the flexibility to trade off the embedding size for a negligible reduction in performance.
 
2786
  For more information, see the [API reference](https://docs.nomic.ai/reference/endpoints/nomic-embed-text)
2787
 
2788
 
2789
+ ## Infinity
2790
+
2791
+ Usage with [Infinity](https://github.com/michaelfeil/infinity).
2792
+
2793
+ ```bash
2794
+ docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7999":"7997" \
2795
+ michaelf34/infinity:0.0.68 \
2796
+ v2 --model-id nomic-ai/nomic-embed-text-v1.5 --revision "main" --dtype float16 --batch-size 8 --engine torch --port 7997 --no-bettertransformer
2797
+ ```
2798
+
2799
  ## Adjusting Dimensionality
2800
 
2801
  `nomic-embed-text-v1.5` is an improvement upon [Nomic Embed](https://huggingface.co/nomic-ai/nomic-embed-text-v1) that utilizes [Matryoshka Representation Learning](https://arxiv.org/abs/2205.13147) which gives developers the flexibility to trade off the embedding size for a negligible reduction in performance.