text
There are 2.2 billion active 
There are 2.2 billion active 
mention
@Apple
Apple
text
 devices 🍏 and all of them just got smarter thanks to Apple Intelligence (AI) 🧠
 devices 🍏 and all of them just got smarter thanks to Apple Intelligence (AI) 🧠
new_line
new_line
text
Well, almost all devices... 🤔
Well, almost all devices... 🤔
new_line
new_line
text
Your device needs:
Your device needs:
new_line
text
- A17 Pro chip or later if it's an iPhone 📱,
- A17 Pro chip or later if it's an iPhone 📱,
new_line
text
- M1 chip or later if iPad 📱,
- M1 chip or later if iPad 📱,
new_line
text
- M1 chip or later if Mac 💻.
- M1 chip or later if Mac 💻.
new_line
new_line
text
All this aside, this is probably the largest deployment of on-device LLMs 🌍.
All this aside, this is probably the largest deployment of on-device LLMs 🌍.
new_line
new_line
text
Here is the technical goodness:
Here is the technical goodness:
new_line
text
- AI will run ~3B LLM on device (Mac, iPhone, iPad) with grouped-query-attention, activation, and embedding quantization (Talaria bit rate selection) running on the neural engine 🚀.
- AI will run ~3B LLM on device (Mac, iPhone, iPad) with grouped-query-attention, activation, and embedding quantization (Talaria bit rate selection) running on the neural engine 🚀.
new_line
text
- Will be using fine-tuned LoRA Adapters for different tasks, claiming to outperform other 7B and 3B LLMs! 🥇
- Will be using fine-tuned LoRA Adapters for different tasks, claiming to outperform other 7B and 3B LLMs! 🥇
new_line
text
- iPhone 15 Pro 0.6 ms time-to-first-token with 30 tokens/second latency ⏱.
- iPhone 15 Pro 0.6 ms time-to-first-token with 30 tokens/second latency ⏱.
new_line
text
- No server model size or details 🤐.
- No server model size or details 🤐.
new_line
text
- Will be dynamically loading, caching, and swapping LoRA adapters (think LoRA Land) 🔄.
- Will be dynamically loading, caching, and swapping LoRA adapters (think LoRA Land) 🔄.
new_line
text
- On-device model has 49K vocab size, while the server model goes 100K 📚.
- On-device model has 49K vocab size, while the server model goes 100K 📚.
new_line
text
- Using rejection sampling fine-tuning and RLHF in post-processing 📈.
- Using rejection sampling fine-tuning and RLHF in post-processing 📈.
new_line
text
- A rejection sampling fine-tuning algorithm with teacher committee 🎓.
- A rejection sampling fine-tuning algorithm with teacher committee 🎓.
new_line
text
- And reinforcement learning from human feedback (RLHF) algorithm with mirror descent policy optimization and a leave-one-out advantage estimator 🧮.
- And reinforcement learning from human feedback (RLHF) algorithm with mirror descent policy optimization and a leave-one-out advantage estimator 🧮.
new_line
text
- Used synthetic data generation (from bigger models, does not mention which) for tasks like summaries 📝.
- Used synthetic data generation (from bigger models, does not mention which) for tasks like summaries 📝.
new_line
text
- 750 evaluation samples for each production use case to evaluate summarization (dataset not released) 📊.
- 750 evaluation samples for each production use case to evaluate summarization (dataset not released) 📊.
new_line
text
- No mention of multilingual support 🌐.
- No mention of multilingual support 🌐.
new_line
text
- Used Apple's AXLearn framework (JAX) and FSP to train on TPUs and GPUs 💪.
- Used Apple's AXLearn framework (JAX) and FSP to train on TPUs and GPUs 💪.
new_line
text
- 3B + Adapter outperforms Phi-3 mini, Gemma 7B, Mistral 7B on summarization 🏆.
- 3B + Adapter outperforms Phi-3 mini, Gemma 7B, Mistral 7B on summarization 🏆.
new_line
text
- 3B + Adapter achieves 78.7% on IFEval beating Phi-3 mini, Gemma 7B, Mistral 7B; Server Model matches GPT-4-Turbo and beats Mixtral 8x22B and GPT-3.5-turbo ✨.
- 3B + Adapter achieves 78.7% on IFEval beating Phi-3 mini, Gemma 7B, Mistral 7B; Server Model matches GPT-4-Turbo and beats Mixtral 8x22B and GPT-3.5-turbo ✨.
new_line
new_line
text
LoRA for the win! 🎉
LoRA for the win! 🎉
new_line
new_line
text
Blog: 
Blog: 
link
https://machinelearning.apple.com/research/introducing-apple-foundation-models
https://machinelearning.apple.com/research/introducing-apple-foundation-models
There are 2.2 billion active @Apple devices 🍏 and all of them just got smarter thanks to Apple Intelligence (AI) 🧠

Well, almost all devices... 🤔

Your device needs:
- A17 Pro chip or later if it's an iPhone 📱,
- M1 chip or later if iPad 📱,
- M1 chip or later if Mac 💻.

All this aside, this is probably the largest deployment of on-device LLMs 🌍.

Here is the technical goodness:
- AI will run ~3B LLM on device (Mac, iPhone, iPad) with grouped-query-attention, activation, and embedding quantization (Talaria bit rate selection) running on the neural engine 🚀.
- Will be using fine-tuned LoRA Adapters for different tasks, claiming to outperform other 7B and 3B LLMs! 🥇
- iPhone 15 Pro 0.6 ms time-to-first-token with 30 tokens/second latency ⏱.
- No server model size or details 🤐.
- Will be dynamically loading, caching, and swapping LoRA adapters (think LoRA Land) 🔄.
- On-device model has 49K vocab size, while the server model goes 100K 📚.
- Using rejection sampling fine-tuning and RLHF in post-processing 📈.
- A rejection sampling fine-tuning algorithm with teacher committee 🎓.
- And reinforcement learning from human feedback (RLHF) algorithm with mirror descent policy optimization and a leave-one-out advantage estimator 🧮.
- Used synthetic data generation (from bigger models, does not mention which) for tasks like summaries 📝.
- 750 evaluation samples for each production use case to evaluate summarization (dataset not released) 📊.
- No mention of multilingual support 🌐.
- Used Apple's AXLearn framework (JAX) and FSP to train on TPUs and GPUs 💪.
- 3B + Adapter outperforms Phi-3 mini, Gemma 7B, Mistral 7B on summarization 🏆.
- 3B + Adapter achieves 78.7% on IFEval beating Phi-3 mini, Gemma 7B, Mistral 7B; Server Model matches GPT-4-Turbo and beats Mixtral 8x22B and GPT-3.5-turbo ✨.

LoRA for the win! 🎉

Blog: https://machinelearning.apple.com/research/introducing-apple-foundation-models
🤝
louisbrulenaudet zzffss SamChaTICE thethinkmachine
/avatars/54483699273ac58a4a6fe1fa4aab65fe.svg
Robert Sinclair
ZeroWw
user