text
Hear, hear, AMD MI300Xs have started to emerge much sooner than expected. 
Hear, hear, AMD MI300Xs have started to emerge much sooner than expected. 
new_line
new_line
text
Here is a 2-part benchmarks report on performing BLOOM-176B inference using 
Here is a 2-part benchmarks report on performing BLOOM-176B inference using 
mention
@MSFTDeepSpeed
MSFTDeepSpeed
text
 optimized for AMD MI300X.
 optimized for AMD MI300X.
new_line
new_line
text
1. 
1. 
link
https://www.evp.cloud/post/diving-deeper-insights-from-our-llm-inference-testing
https://www.evp.cloud/post/diving-deeper-insights-from-our-llm-inference-testing
new_line
text
2. 
2. 
link
https://www.evp.cloud/post/diving-deeper-insights-from-our-llm-inference-testing-part-2
https://www.evp.cloud/post/diving-deeper-insights-from-our-llm-inference-testing-part-2
new_line
new_line
text
This was published in response to our BLOOM-176B super-fast inference blog post 
This was published in response to our BLOOM-176B super-fast inference blog post 
link
https://huggingface.co/blog/bloom-inference-pytorch-scripts
https://huggingface.co/blog/bloom-inference-pytorch-scripts
text
new_line
new_line
text
Note that these have 192GB of HBM!
Note that these have 192GB of HBM!
new_line
new_line
text
The NVIDIA monopoly is strong, but it'll have to start sharing the pie and hopefully drive the costs down at least somewhat.
The NVIDIA monopoly is strong, but it'll have to start sharing the pie and hopefully drive the costs down at least somewhat.
new_line
new_line
text
Thanks to 
Thanks to 
link
https://www.linkedin.com/in/eliovp
https://www.linkedin.com/in/eliovp
text
 for sharing this writeup with me.
 for sharing this writeup with me.
new_line
new_line
text
p.s. at the PyTorch conference in the fall, the AMD representative said we will see MI300X available to us mortals in Q4-2024/Q1-2025.
p.s. at the PyTorch conference in the fall, the AMD representative said we will see MI300X available to us mortals in Q4-2024/Q1-2025.
Hear, hear, AMD MI300Xs have started to emerge much sooner than expected. 

Here is a 2-part benchmarks report on performing BLOOM-176B inference using @MSFTDeepSpeed optimized for AMD MI300X.

1. https://www.evp.cloud/post/diving-deeper-insights-from-our-llm-inference-testing
2. https://www.evp.cloud/post/diving-deeper-insights-from-our-llm-inference-testing-part-2

This was published in response to our BLOOM-176B super-fast inference blog post https://huggingface.co/blog/bloom-inference-pytorch-scripts 

Note that these have 192GB of HBM!

The NVIDIA monopoly is strong, but it'll have to start sharing the pie and hopefully drive the costs down at least somewhat.

Thanks to https://www.linkedin.com/in/eliovp for sharing this writeup with me.

p.s. at the PyTorch conference in the fall, the AMD representative said we will see MI300X available to us mortals in Q4-2024/Q1-2025.
❤️
osanseviero julien-c davanstrien tomaarsen VictorSanh clem pcuenq muhtasham merve samusenps Eliovp andrewrreed rwightman clefourrier
https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/NQtzmrDdbG0H8qkZvRyGk.jpeg
Julien Chaumond
julien-c
user
https://cdn-avatars.huggingface.co/v1/production/uploads/6317233cc92fd6fee317e030/cJHSvvimr1kqgQfHOjO5n.png
Tom Aarsen
tomaarsen
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1583857146757-5e67bdd61009063689407479.jpeg
Clem 🤗
clem
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1594311341799-5f07383b19cb630495b812cd.jpeg
Stas Bekman
stas
user
/avatars/bc9d6b4976a2006fd53ce27b55d776d6.svg
Eliovp
Eliovp
user