FP8 KV Cache - a mohitsha Collection

mohitsha 's Collections

AMD RyzenAI Models

FP8 KV Cache

updated Jul 4, 2024

Models with FP8 KV Cache Scales

mohitsha/Llama-2-70b-chat-hf-FP8-KV

Text Generation • Updated Jun 25, 2024 • 19
mohitsha/Llama-2-7b-chat-hf-FP8-KV

Text Generation • Updated Jun 25, 2024 • 27
mohitsha/Llama-2-7b-chat-hf-FP8-KV-AMMO

Text Generation • Updated Jun 25, 2024 • 24
mohitsha/Llama-2-70b-chat-hf-FP8-KV-AMMO

Text Generation • Updated Jun 25, 2024 • 22
mohitsha/Llama-2-7b-chat-hf-FP8-FNUZ-KV-AMMO

Text Generation • Updated Jun 25, 2024 • 21