text
It took Google’s Transformer model from 2017 a whopping $900 to train. 💸
It took Google’s Transformer model from 2017 a whopping $900 to train. 💸
new_line
new_line
text
This in contrast to the $191 million Google spent on Gemini Ultra sounds like a bargain! 💰
This in contrast to the $191 million Google spent on Gemini Ultra sounds like a bargain! 💰
new_line
new_line
text
Gemini Ultra required 50 billion petaFLOPS (one petaFLOP equals one quadrillion FLOPs). 🤖 
Gemini Ultra required 50 billion petaFLOPS (one petaFLOP equals one quadrillion FLOPs). 🤖 
new_line
text
Compared to OpenAI’s GPT-4, which required 21 billion petaFLOPS, at a cost of $78 million. 💡
Compared to OpenAI’s GPT-4, which required 21 billion petaFLOPS, at a cost of $78 million. 💡
new_line
new_line
text
2017: Original Transformer Model: $930 [@Google ] 💻  
2017: Original Transformer Model: $930 [@Google ] 💻  
new_line
text
2018: BERT-Large: $3,288 [@Google] 📚  
2018: BERT-Large: $3,288 [@Google] 📚  
new_line
text
2019: RoBERTa Large: 160k [@Meta] 🌐  
2019: RoBERTa Large: 160k [@Meta] 🌐  
new_line
text
2020: GPT-3(175B): $4.32M [@OpenAI] 🧠  
2020: GPT-3(175B): $4.32M [@OpenAI] 🧠  
new_line
text
2023: Llama 2 70B: $3.93M [@Meta] 🐑  
2023: Llama 2 70B: $3.93M [@Meta] 🐑  
new_line
text
2023: GPT-4: $78.35M [@OpenAI] 🌟  
2023: GPT-4: $78.35M [@OpenAI] 🌟  
new_line
text
Now, Gemini Ultra: $191.4M [@Google] 🚀  
Now, Gemini Ultra: $191.4M [@Google] 🚀  
new_line
new_line
text
This forms an exponential curve! 🤯
This forms an exponential curve! 🤯
new_line
new_line
text
But, why? 🤔  
But, why? 🤔  
new_line
text
Compute, data, and expertise. All three come at a great cost! ⚙️📊💡
Compute, data, and expertise. All three come at a great cost! ⚙️📊💡
new_line
new_line
text
Google recently made Gemini-1.5-Flash fine-tuning free, as it's almost impossible for regular businesses to justify an in-house trained foundational model! 🆓
Google recently made Gemini-1.5-Flash fine-tuning free, as it's almost impossible for regular businesses to justify an in-house trained foundational model! 🆓
new_line
new_line
text
This barrier of cost is going to result in fewer new foundational 
This barrier of cost is going to result in fewer new foundational 
text
models/less
models/less
text
 competition and more fine-tunes! 📉🔄
 competition and more fine-tunes! 📉🔄
new_line
new_line
text
Data [Stanford University’s 2024 AI Index Report]: 
Data [Stanford University’s 2024 AI Index Report]: 
link
https://aiindex.stanford.edu/report/
https://aiindex.stanford.edu/report/
text
new_line
text
Graphic: 
Graphic: 
link
https://voronoiapp.com/technology/Googles-Gemini-Ultra-Cost-191M-to-Develop--1088
https://voronoiapp.com/technology/Googles-Gemini-Ultra-Cost-191M-to-Develop--1088
new_line
new_line
text
Many thanks to everyone spending tons of resources and open-sourcing the models! 🤗
Many thanks to everyone spending tons of resources and open-sourcing the models! 🤗
It took Google’s Transformer model from 2017 a whopping $900 to train. 💸

This in contrast to the $191 million Google spent on Gemini Ultra sounds like a bargain! 💰

Gemini Ultra required 50 billion petaFLOPS (one petaFLOP equals one quadrillion FLOPs). 🤖 
Compared to OpenAI’s GPT-4, which required 21 billion petaFLOPS, at a cost of $78 million. 💡

2017: Original Transformer Model: $930 [@Google ] 💻  
2018: BERT-Large: $3,288 [@Google] 📚  
2019: RoBERTa Large: 160k [@Meta] 🌐  
2020: GPT-3(175B): $4.32M [@OpenAI] 🧠  
2023: Llama 2 70B: $3.93M [@Meta] 🐑  
2023: GPT-4: $78.35M [@OpenAI] 🌟  
Now, Gemini Ultra: $191.4M [@Google] 🚀  

This forms an exponential curve! 🤯

But, why? 🤔  
Compute, data, and expertise. All three come at a great cost! ⚙️📊💡

Google recently made Gemini-1.5-Flash fine-tuning free, as it's almost impossible for regular businesses to justify an in-house trained foundational model! 🆓

This barrier of cost is going to result in fewer new foundational models/less competition and more fine-tunes! 📉🔄

Data [Stanford University’s 2024 AI Index Report]: https://aiindex.stanford.edu/report/  
Graphic: https://voronoiapp.com/technology/Googles-Gemini-Ultra-Cost-191M-to-Develop--1088

Many thanks to everyone spending tons of resources and open-sourcing the models! 🤗
🔥
Pavun Felladrin Riswan-Nopiyar paulaju thisisanshgupta Abduraxim NHLOCAL prithivMLmods awhsnap kmsky
👍
Falln87 SyedRehman110 NHLOCAL
😎
wonderboy
https://cdn-avatars.huggingface.co/v1/production/uploads/623aada982038110d90921f3/nSHSXJz7PYyw9fn8qoAn4.jpeg
zero
wonderboy
user
/avatars/056370e376f77d75e8092f0c6338956f.svg
Nithin I Bhandari
nib12345
user
https://cdn-avatars.huggingface.co/v1/production/uploads/66910eabeabb133027131370/ZCRNJNuk6yOgSyrWLHkcU.jpeg
leonardo zi
leonardozi
user
/avatars/744eddaa7dfc34a57df9ce32a78059a0.svg
Tyrone Pierce
piercyy
user