text
🏅 Quantized Embeddings are here! Unlike model quantization, embedding quantization is a post-processing step for embeddings that converts e.g. 
🏅 Quantized Embeddings are here! Unlike model quantization, embedding quantization is a post-processing step for embeddings that converts e.g. 
inline_code
`float32`
float32
text
 embeddings to binary or 
 embeddings to binary or 
inline_code
`int8`
int8
text
 embeddings. This saves 32x or 4x memory & disk space, and these embeddings are much easier to compare!
 embeddings. This saves 32x or 4x memory & disk space, and these embeddings are much easier to compare!
new_line
new_line
text
Our results show 25-45x speedups in retrieval compared to full-size embeddings, while keeping 96% of the performance!
Our results show 25-45x speedups in retrieval compared to full-size embeddings, while keeping 96% of the performance!
new_line
new_line
text
Learn more about it in our blogpost in collaboration with mixedbread.ai: 
Learn more about it in our blogpost in collaboration with mixedbread.ai: 
link
https://huggingface.co/blog/embedding-quantization
https://huggingface.co/blog/embedding-quantization
new_line
text
Or try out our demo where we use quantized embeddings to let you search all of Wikipedia (yes, 41,000,000 texts) in 1 second on a CPU Space: 
Or try out our demo where we use quantized embeddings to let you search all of Wikipedia (yes, 41,000,000 texts) in 1 second on a CPU Space: 
resource
https://huggingface.co/spaces/sentence-transformers/quantized-retrieval
space
sentence-transformers/quantized-retrieval
https://huggingface.co/spaces/sentence-transformers/quantized-retrieval
🏅 Quantized Embeddings are here! Unlike model quantization, embedding quantization is a post-processing step for embeddings that converts e.g. `float32` embeddings to binary or `int8` embeddings. This saves 32x or 4x memory & disk space, and these embeddings are much easier to compare!

Our results show 25-45x speedups in retrieval compared to full-size embeddings, while keeping 96% of the performance!

Learn more about it in our blogpost in collaboration with mixedbread.ai: https://huggingface.co/blog/embedding-quantization
Or try out our demo where we use quantized embeddings to let you search all of Wikipedia (yes, 41,000,000 texts) in 1 second on a CPU Space: https://huggingface.co/spaces/sentence-transformers/quantized-retrieval
🔥
jaggernaut007 snowdere osanseviero AtAndDev cnmoro mohammedbriman clem pascalnjue louisbrulenaudet iamrobotbear mrdbourke danielpark MichelleRuwen
❤️
samusenps osanseviero MarklarFromMarklar AtAndDev clem AiModelsMarket iamrobotbear EllenGu danielpark MichelleRuwen
/avatars/b2b429dd81d366b42a74a36f8e237286.svg
D
snowdere
user