text
Last week, we released 🤗 Transformers.js v2.14, which added support for SAM (Segment Anything Model).
Last week, we released 🤗 Transformers.js v2.14, which added support for SAM (Segment Anything Model).
new_line
new_line
text
This means you can now generate high-quality segmentation masks for objects in a scene, directly in your browser! 🤯
This means you can now generate high-quality segmentation masks for objects in a scene, directly in your browser! 🤯
new_line
new_line
text
Demo (+ source code): 
Demo (+ source code): 
resource
https://huggingface.co/spaces/Xenova/segment-anything-web
space
Xenova/segment-anything-web
https://huggingface.co/spaces/Xenova/segment-anything-web
new_line
text
Model: 
Model: 
resource
https://huggingface.co/Xenova/slimsam-77-uniform
model
Xenova/slimsam-77-uniform
https://huggingface.co/Xenova/slimsam-77-uniform
new_line
new_line
text
But how does this differ from Meta's original demo? 🤔 Didn't that also run in-browser?
But how does this differ from Meta's original demo? 🤔 Didn't that also run in-browser?
new_line
new_line
text
Well, in their demo, the image embeddings are computed server-side, then sent to the client for decoding. Trying to do this all client-side would be completely impractical: taking minutes per image! 😵‍💫
Well, in their demo, the image embeddings are computed server-side, then sent to the client for decoding. Trying to do this all client-side would be completely impractical: taking minutes per image! 😵‍💫
new_line
new_line
text
That's where SlimSAM comes to the rescue! SlimSAM is a novel SAM compression method, able to shrink the model over 100x (637M → 5.5M params), while still achieving remarkable results!
That's where SlimSAM comes to the rescue! SlimSAM is a novel SAM compression method, able to shrink the model over 100x (637M → 5.5M params), while still achieving remarkable results!
new_line
new_line
text
The best part? You can get started in a few lines of JavaScript code, thanks to Transformers.js! 🔥
The best part? You can get started in a few lines of JavaScript code, thanks to Transformers.js! 🔥
new_line
new_line
code_fence
```
// npm i @xenova/transformers
import { SamModel, AutoProcessor, RawImage } from '@xenova/transformers';

// Load model and processor
const model = await SamModel.from_pretrained('Xenova/slimsam-77-uniform');
const processor = await AutoProcessor.from_pretrained('Xenova/slimsam-77-uniform');

// Prepare image and input points
const img_url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/corgi.jpg';
const raw_image = await RawImage.read(img_url);
const input_points = [[[340, 250]]];

// Process inputs and perform mask generation
const inputs = await processor(raw_image, input_points);
const outputs = await model(inputs);

// Post-process masks
const masks = await processor.post_process_masks(outputs.pred_masks, inputs.original_sizes, inputs.reshaped_input_sizes);
console.log(masks);

// Visualize the mask
const image = RawImage.fromTensor(masks[0][0].mul(255));
image.save('mask.png');
```
// npm i @xenova/transformers
import { SamModel, AutoProcessor, RawImage } from '@xenova/transformers';

// Load model and processor
const model = await SamModel.from_pretrained('Xenova/slimsam-77-uniform');
const processor = await AutoProcessor.from_pretrained('Xenova/slimsam-77-uniform');

// Prepare image and input points
const img_url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/corgi.jpg';
const raw_image = await RawImage.read(img_url);
const input_points = [[[340, 250]]];

// Process inputs and perform mask generation
const inputs = await processor(raw_image, input_points);
const outputs = await model(inputs);

// Post-process masks
const masks = await processor.post_process_masks(outputs.pred_masks, inputs.original_sizes, inputs.reshaped_input_sizes);
console.log(masks);

// Visualize the mask
const image = RawImage.fromTensor(masks[0][0].mul(255));
image.save('mask.png');
new_line
new_line
text
I can't wait to see what you build with it! 🤗
I can't wait to see what you build with it! 🤗
Last week, we released 🤗 Transformers.js v2.14, which added support for SAM (Segment Anything Model).

This means you can now generate high-quality segmentation masks for objects in a scene, directly in your browser! 🤯

Demo (+ source code): https://huggingface.co/spaces/Xenova/segment-anything-web
Model: https://huggingface.co/Xenova/slimsam-77-uniform

But how does this differ from Meta's original demo? 🤔 Didn't that also run in-browser?

Well, in their demo, the image embeddings are computed server-side, then sent to the client for decoding. Trying to do this all client-side would be completely impractical: taking minutes per image! 😵‍💫

That's where SlimSAM comes to the rescue! SlimSAM is a novel SAM compression method, able to shrink the model over 100x (637M → 5.5M params), while still achieving remarkable results!

The best part? You can get started in a few lines of JavaScript code, thanks to Transformers.js! 🔥

```
// npm i @xenova/transformers
import { SamModel, AutoProcessor, RawImage } from '@xenova/transformers';

// Load model and processor
const model = await SamModel.from_pretrained('Xenova/slimsam-77-uniform');
const processor = await AutoProcessor.from_pretrained('Xenova/slimsam-77-uniform');

// Prepare image and input points
const img_url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/corgi.jpg';
const raw_image = await RawImage.read(img_url);
const input_points = [[[340, 250]]];

// Process inputs and perform mask generation
const inputs = await processor(raw_image, input_points);
const outputs = await model(inputs);

// Post-process masks
const masks = await processor.post_process_masks(outputs.pred_masks, inputs.original_sizes, inputs.reshaped_input_sizes);
console.log(masks);

// Visualize the mask
const image = RawImage.fromTensor(masks[0][0].mul(255));
image.save('mask.png');
```

I can't wait to see what you build with it! 🤗
❤️
osanseviero radames rahuldshetty victor brunnolou samusenps afrideva Dlbk clem rreed zuozuo AdamSako omaryshchenko NikitaKras chinmayak cggaurav bmullan doublelotus horizon0514
👍
smalltong02 mlabonne Sylvestre clem rreed Norod78 AkindOf
🤯
osanseviero rahuldshetty clem rreed
🤗
rreed
/avatars/ffbed556c373d95be74c2b2daf7ac09d.svg
zhang hong
crapthings
user
/avatars/7b61bed87fcf7a66ad4329653f0937a0.svg
Gaurav C
cggaurav
user
https://cdn-avatars.huggingface.co/v1/production/uploads/61b253b7ac5ecaae3d1efe0c/hwiQ0uvz3t-L5a-NtBIO6.png
Joshua
Xenova
user
/avatars/a0ca8cfc453d3b04603362e50de8265d.svg
Jeremiah M
doublelotus
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1637059166787-60389fb792aa59eee1bca329.jpeg
Matthieu Vanhoutte
Matthieu
user
/avatars/eae6ec62924637479836a654d004739b.svg
Rithesh Sreenivasan
ritheshSree
user
/avatars/5df778a0a4cce6ee7b543830b9cf0e48.svg
Pjotter
pjotterp
user